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Foreword 



Management education is a booming business. More and more consultants are moving into 
this area, forcing business schools to defend their traditional territory. American schools are 
allying themselves with institutions in other parts of the world and setting up subsidiaries, 
joint ventures and strategic alliances to leverage their know-how. On the business side, the 
number of Corporate Universities - establishments dedicated to management development - 
is growing rapidly, even in Europe. In the “battle to win talent”, companies want to show that 
they provide their top executives with the best chances of success. 

However, the picture is not all rosy. These days every expense has to prove its value added, 
and management education is no exception. But as this analysis shows, a more in-depth 
evaluation than the “happy-sheet” survey handed in at the end of a programme is difficult to 
compile. First one needs to ask the question, “Who is the evaluation for, e.g., the accounts 
department, HR, the participants or their superiors?” Another tricky question is, “Are you 
measuring HR indicators or programme learning goals?” It is not at all clear that learning 
goals, even for customised programmes, are tied to companies’ “normal" HR evaluation 
process (often in the form of a balanced scorecard) and the related incentive schemes (the 
reason for this would require a thesis on its own). 

Furthermore, as this analysis has revealed, there are at least 12 different concepts of “how” 
to evaluate. But is it really worth the effort? Can’t we assume that grown managers can take 
care of their own education, take what they need and alert their HR people if their time has 
been wasted? And last but not least, how can the evaluation be worded in such a way that it 
is short and straightforward enough for busy managers to be bothered to answer, but still 
allows one to gather useful, relevant information? 

But these are just the sorts of challenges that a business school relishes. One has to 
understand that measuring the benefits of management education is much more of a multi- 
dimensional exercise, and that the results are only the beginning of a meaningful dialogue 
between the stakeholders involved - HR, sponsors, accountants, participants, etc. - leading 
to a better alignment of development goals and delivery. This calls for a “balanced scorecard 
approach”, as described in this excellent contribution, and a process driver, who makes sure 
that the information thus acquired is acted upon. But all the difficulties explain why there is 
huge support for a better evaluation of management education in general but an obvious 
reluctance to tackle these difficulties head on and improve the evaluation system through 
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more experimentation and testing of new tools. This thesis demonstrates how this can be 
done in the framework of one of the most global, and hence complex, companies. 

Has this research revealed any surprises? At a first glance, there are surprisingly few 
learning differences between the “tailor-made”, company-specific programmes and the open 
general management programmes. Two explanations are possible: first, business schools do 
not really “tailor” programmes, but just repackage them using “off-the-shelf modules. There 
are clearly differences between business schools in this respect, often depending on their 
business model and size, but I doubt that this is the main reason. More plausible for me is 
the second interpretation, which is that managers selectively pick “nuggets” from whatever 
programme they are on, and “twist” the knowledge to fit their needs. After all, the basic 
process of management development remains the same, i.e., that programmes offer a 
platform (and time-out) to reflect and to conceptualise a rich, but often fragmented and 
intuitive, knowledge base. Equipped with a better understanding, based on frameworks and 
accepted experience about what works and what does not, participants emerge with their 
newly acquired knowledge, ready to face any challenge. 

But this is as fundamental - and generic - as the notion that companies have more to earn 
than they spend. We need to know more about the specifics, which is why this thesis is an 
extremely useful basis for evaluation for every Corporate University or HR department. 



Ulrich Steger 
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1 Introduction and Overview 



“In the 21 st century , the education and skills of the workforce will end up being the 
dominant competitive weapon. " 



(Thurow 1992, p. 275) 

The success or failure of corporations in today’s global economy depends on many 
individual, organisational and environmental variables. However, one of the most important 
factors for sustaining competitive success are the people in an organisation. Practitioners 
and academics alike agree, that intellectual capital has become the new basis for 
competition in the post-capitalist society (Drucker 1993). Individual and organisational 
knowledge plays thus a central role in building competitive advantage (Hamel & Prahalad 
1994b). The ability to leam and to un-learn has become a critical - if not the most important - 
factor for ensuring corporate success in a world of growing complexity and ever faster 
technological changes (Argyris & Schon 1978; Senge 1990; Davenport & Prusak 1998). 

The human resource function can contribute to a corporation’s competitive advantage by 
acquiring and developing the necessary human capital, i.e. individual and organisational 
capabilities. On the one hand, the human resource function can attract and select the right 
people from the outside and thus acquire the capabilities demanded. On the other hand, it 
can develop existing human capital in order to build capabilities, which strengthen the 
competitive position of the organisation. 

In the framework of our thesis, we take a look at the latter way of building and developing 
individual and organisational capabilities. Many organisations today use a variety of human 
resource development methods to enhance their human talent. These may include on-the- 
job measures, like international assignments or coaching, and off-the-job interventions, like 
skill training and developmental programmes. 

Many companies have recognised the importance of human resource development and are 
thus investing an increasing share of their funds in the training of their human capital. The 
spending on corporate education in the United States alone developed from $43.2 billion in 
1991 to $60.7 billion in 1998 according to a Merrill Lynch report (Moe et al. 1999, p. 134). 
Companies have created new positions, for example the chief learning officer , and have 
established new forms of corporate education, for example corporate universities. As a 
result, the area of corporate training and development has been characterised by rapid 
growth and innovation over the last decade. The following table outlines the historic 
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development of training and illustrates the increasing importance of training and development 
in today’s global economy. 





Craft 

System 


Market 

System 


Technical 

System 


Career 

System 


Human 

Resource 

System 


Virtual 

System 


Timeframe 


Middle Ages 


Industrialisation 


Inter-war Period 


Post-war Period 


Computer Age 1 


2 

Internet Age 


Work 

Organisation 


Master/ 

journeymen/ 

apprentice 


Unskilled 

machine 

operators 


Fine division of 
unskilled labour 
with machine 
pacing 


Groups of 
employees 
under a 
supervisor 


Work teams, no 
supervisor 


Changing 

network 

structures 


Employer’s 
Perception of 
Employees 


Apprentices 
often viewed as 
“sons" 


Skilled and 
unskilled viewed 
as variable 
costs 


Viewed as being 
interchangeable 


Long-term, 

career- 

orientated 

community 

members 


Core vs. 
contingent 
“contract" 
employees 


Intellectual 

capital 


Length of 
Service 


Long-term 


Based on 

market 

fluctuation 


Semi-long term, 
layoff by 
seniority 


Employment 

stability 


Less 

guaranteed 


Dynamic and 
short-lived 


Role of the 
Manager 


Patriarch 


Guard 


Bureaucrat 


Supervisor 


Team facilitator 


Change agent 


Focus of 
Training and 
Development 


Individual 
employee needs 


Fewer skills 
taught to fewer 
people 


Functional 
expertise and 
administrative 
efficiency 
stressed 


Teams, 

technical skills, 
and retaining 
stressed 


“Employability” 
and continuous 
learning 
stressed 


Knowledge 

management, 

core 

competencies 


Status of 
Training and 
Development 


Central to 
business 


Decreasingly 

needed 


Obsolete 


A growing value 


A core value 


A strategic 
value/ source of 
competitive 
advantage 



Table 1.1: The Evolution of Training and Development 

Derived from Ingols & Sonnenfeld 1986, pp. 63-67 and amended 



The theme of corporate education provides the broad frame for our thesis. In our study we 
are interested in one particular form of training and development. We investigate the value of 
face-to-face executive education programmes, which are offered internally through a 
corporate university. This traditional form of education has recently gained a new 
dimension through an intensified co-operation of corporate universities and business 
schools. 

The aspect of customising business school executive education to one’s own strategic 
needs represents a new development in the area of Human Resource Management 
(Ghoshal et al. 1992). However, very little research has been done to understand the value 
of these customised executive programmes due to a lack of appropriate research tools 



1 The “Computer Age” began in the mid 1970s and lasted until the arrival of the internet in the mid 1990s. 

2 The “Internet Age" started in the mid 1990s and has lasted until today. 
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(Liedtka et a\. 1999). The missing empirical research on these programmes has been the 
starting point for our thesis. 

In order to provide a more detailed orientation on our study, we outline the research context, 
our research objectives and assumptions in the following parts. Subsequently, we close the 
introduction with an overview of the structure of our thesis. 



1.1 The Research Context 

“The primary objective of DCU’s customised executive education programmes 
is not to build technical competence, but to provide general education on 
strategy. We want to teach executives how to define and implement strategy 
(1) faster, (2) better and (3) more effectively/efficiently.” 

(Senior Manager of the DaimlerChrysler Corporate University 1999; emphasis added) 

First of all, it is important to mention that our research has been initiated and funded by the 
DaimlerChrysler Corporate University (DCU). The close co-operation with the DCU has 
offered a unique research opportunity to our research and has therefore assisted us in 
defining our research focus. Given our interest in executive education, three broad thematic 
fields delineate the research area of our thesis. Our research focus rests on face-to-face 
executive programmes, which are located in the intersection of the fields of Human Resource 
Development (HRD), corporate strategy and corporate universities. 



W = Area of Research Customised 

Executive Education 



Strategic 

Human 

Resource 

Development 




Evaluation of 
Face-to-face 
Executive 
Programmes 



Figure 1.1: The Thesis’ Area of Research 
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Figure 1.1 (on the previous page) represents a schematic and simplified illustration of the 
context of our research. Thereby, it is important to note that executive education represents 
only one element among others, which is found in the intersection of HRM , corporate 
strategy and corporate universities. For example, the intersection can comprise general 
training for all employees as well. Furthermore, executive education is not only found in the 
specified intersection. It can take place independently from a corporate university. The 
purpose of figure 1.1 is to introduce the four cornerstones of our research context. We have 
chosen the four cornerstones for the following reasons. 

First of all, corporate universities represents the broad framework of our research. We 
investigate those management development programmes 3 , which are part of the activity 
portfolio of corporate universities. Executive programmes represent only one activity in the 
portfolio of corporate universities. In most cases, different developmental human resource 
methods are combined under the corporate university umbrella. 

The second cornerstone of our research context consists of the aspect of customisation. 
Very little academic work has been done on the customisation of business school executive 
education in the past. Customisation presents thus a new research field, which motivated us 
to contribute to its understanding. 

The strategic content of executive education embodies the third cornerstone. The last ten 
years has seen an increase in educational demand in strategic management and has given 
rise to Strategic Human Resource Development (SHRD). Teaching strategic thinking is the 
common denominator for most executive programmes. 

Finally, the most important cornerstone is the evaluation of these executive programmes. 
Given the recent developments in executive education, there is a visible lack of practical 
approaches that assess the value of customised face-to-face executive programmes. 

In the following sections, we outline the four cornerstones in more detail in order to provide 
the reader with a better comprehension of our research context. Thereby, we aim at creating 
a common ‘platform of understanding’ on which we can place our research objectives later 
on. 



3 In our thesis we use the terms “executive education programmes" and “management development programmes” 
interchangeably. 



4 




1 .1 .1 The Phenomenon of Corporate Universities 



“A corporate university is a centralised in-house training and education facility to 
address the shortened shelf life of knowledge and to align training and 
development with business strategies . " 



(Meister 1998b, p. 38) 

In line with the development of the human resource function, the last 15 years saw a 
significant growth in corporate universities. Estimates state that the number of corporate 
universities rose from 400 in 1988 to ca. 1,600 in 1998 (Strauss 1998). In a recent survey, 
over 400 of the Fortune 500 corporations “were planning or had already instituted a 
corporate university ” (Wagner 2000, p. 93). Ca. 40 percent of the Fortune 500 firms have 
already established a corporate university 4 (see appendix 8.1 for a short description of the 
origins of corporate universities). The recent growth in numbers of corporate universities has 
attracted academic discussion and has created an awareness of the phenomenon. Whereas 
in the past, most corporate universities were found in North American corporations, there has 
been a recent proliferation of corporate universities among German companies as well 
(Andresen & Irmer 1999; Huy & Lang 2000; Hilse 2001). In the last five years the 
management literature started to analyse more intensively the specific characteristics of 
those in-house universities in Germany. However, specific research on the field of German 
corporate universities has remained limited until today. 

The definitions of corporate universities in the literature vary. Meister (1998a), one of the 
pioneers in the field, discerns a clear distinction between a corporate university and previous 
training departments. In her view, traditional training departments were decentralised and 
reactive. Furthermore, they focused primarily on instructing internal employees in job skills. 
In contrast, 

"... a corporate university is the chief vehicle for disseminating an organisation’s 
culture and fostering the development of not only job skills, but also such core 
workplace skills as learning-to-learn, leadership, creative thinking, and problem 
solving . " 



(Meister 1998b, p. 38) 



4 The figures have to be considered with caution. The survey does not distinguish between ‘colleges’, ‘corporate 
universities’, 'institutes' or ‘education centres’. Therefore, distortions might have arisen by combining different 
concepts which have merely similar names. 
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In comparison to previous training and development concepts, one of the significant novelties 
is the emphasis on the strategic role of corporate universities. Moore (1997) illustrates the 
integration of corporate strategy and corporate universities. 

" The best corporate universities share the following characteristics: Goals linked 
to key corporate strategic imperatives , training based on sophisticated 
competency models , development of a shared vision in the corporation as a key 
objective; and function as a laboratory for creating and transforming individual 
learning into organisational knowledge. ’’ 



(Moore 1997, p. 78) 

Deiser (1997) provides a typology that classifies the “ideal types" of corporate universities. In 
table 1.2 we can see Deiser’s description of the four pure forms of corporate universities. 



Type of CU 


Key Function 


Business Logic 


Role of Top 
Management 


Typical Example 


Profit Centre 


Source of Revenue 


Independent 
Business Unit 


Supervisory Board 


Outsourced Training 
Departments 


Qualification 

Centre 


Transfer of Critical 
Skills and 
Knowledge 


Innovation 

Quality 


Allocation of 
Resources 


Motorola 


Standardisation 

Engine 


Transfer of Core 
Practices 
“Enculturation” 


Economies of Scale 


Definition of 
Standards 


Anderson Consulting 
McDonald’s 
Disney 


Platform for 
Strategic Change 


Supporting 
Transformation 
Processes 
“Strategic Learning" 


Critical Tool for 
Corporate 
Leadership 


Architect and Leader 
of Change 


DaimlerChrysler 

IBM 

General Electric 



Table 1 .2: Ideal Types of Corporate Universities 

Source: Deiser 1997, p. 44 5 



Deiser’s classification plays a fundamental role for the development of the research direction 
of our thesis. Deiser’s fourth type, the platform for strategic change , demonstrates the close 
link to Strategic Human Resource Development (SHRD). A corporate university is seen as a 
corporate instrument to support the making and the implementation of corporate strategy. As 



5 The typology includes two alterations that Deiser added in his later function as the Dean of the DaimlerChrysler 
Corporate University. 



6 




a result, the executive programmes, which form part of a corporate university, possess a 
strategic focus as well. These programmes are of interest in our thesis. 



1.1.2 The Aspect of Customisation in Executive Education 

“Increasingly, it would appear that organisations are seeking to work in 
partnership with suppliers of corporate management education. This is leading to 
organisations not only wanting to play a major part in the design of a programme 
with the provider, but increasingly having some involvement in the delivery of the 
programme as well . " 



(Prince & Stewart 2000, p. 211) 

Most corporate universities possess a broad portfolio of training and development activities. 
Some include organisational functions that extend the area of HRM, as for example 
corporate communication (see figure 1.2). In our thesis we focus our research on the aspect 
of face-to-face executive programmes, which account for a significant share within the 
range of activities of corporate universities (Hackett 1997). However, as we can see in figure 
1.2, face-to-face executive programmes are only one element in the activity portfolio of a 
corporate university. Furthermore, strategic management is only one content focus of a 
corporate university (see section 1.1.3). 



Content 



Methods 





Figure 1 .2 The Research Focus within the Corporate University Concept 
Derived from Catry 2001 , unpublished illustration 
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These management development programmes are either run internally or they are delivered 
by business schools (Wood 2001). The latter programmes are tailored, or customised 
(Stiefel 2000), to the specific needs of the corporate university and the company. 

Customisation represents a recent trend in executive education (Bradshaw, FT Survey on 
Executive Education 2001). It implies that the corporate university partners with a business 
school to design an executive programme, which suits the content requirements of the 
company. The corporate university thereby integrates the business school’s expertise and 
teaching services into its programme. As a result, the programme is given by the professors 
of the business school and contains customised content with respect to the current 
challenges of the company. In most cases, the programme takes place at the campus of the 
business school. 6 In some cases, an external hotel or conference centre serves as the 
teaching location. 

The main criticism towards these programmes claims that the degree to which the content is 
customised is too low and that business schools use these programmes as an easy source 
of revenues. We try to respond to these criticism in our thesis as customisation represents 
a further area of our research. We aim to contribute to the current debate on the value of 
customising executive education in the framework of a corporate university. 



1.1.3 The Increase in Educational Demand for Strategic Management 

“In a world of increasing competition , a business leader today must also be a 
strategist. This requires more than just the knowledge of strategic business 
concepts. The objective is to develop superior skills in competitive strategic 
thinking. ” 



(Larreche 2000, p. 1) 

Another element in the context of our research is the increasing demand to teach strategic 
management in executive education. The importance of learning how to make and 
implement strategy, i.e. the strategy process (Mintzberg & Quinn 1991) can be seen in figure 
1.1, which lists the content trends for executive education. Strategic management ranks 
second in the request for topics in management development. We can see that the art of 



6 Most business schools have designated accommodation capacity for their executive programmes or have close 
ties with a hotel nearby. 
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strategy-making and implementation plays an important role in the overall demand for 
executive education. 



Content Trends in Demand Ranked From One to Five 

Change Management 
Strategic Mangement 
Globalisation 
Management of Innovation 
Leadership 
Project Management 
Marketing/ Sales Management 
Personality Management 
Intercultural Management 
Personnel Management 

0 1 2 3 4 5 

1 = Decreasing Demand to 5 = Increasing Demand 

Figure 1.3: Trends in the Demand for Content in Executive Education 

Source: Voss 1999, p. 19 
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The growing interest of integrating strategy into training and development can also be seen 
in the emergence of Strategic Human Resource Management (SHRM), and more specifically 
Strategic Human Resource Development (SHRD). Both approaches emphasise the 
importance of aligning human resource activities with corporate strategy. The strategic 
content and orientation is one of the key characteristic of our research area. Corporate 
universities and customised executive education delineate the broader field of research. 
Therein, the aspect of teaching strategic management is embedded and thus represents 
another focus of our thesis. 



1.1.4 The Need for Evaluation 

“A review of both the academic and practitioners literatures yielded a 
disappointing dearth of empirical work assessing the effectiveness of executive 
education. ’’ 



(Liedtka etal. 1999, p. 405) 
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In the area of business education, a primary concern of academics and practitioners today is 
the aspect of evaluating the effects and impact of management training and development 
(Bassi & McMurrer 1999). Companies started to think about the return of investment of 
education. 

“ For employers, the value of training ultimately lies in the bottom line. ... While 
dozens of studies reveal the economic value of training, expressing such 
improvements in precise dollar amounts that can be attributed solely to training is 
extremely difficult. ” 



(Hackett 1997, p.17) 

The difficulty of evaluation has led to a dilemma in the field of executive education. 
Companies have stepped up their activities of training and developing their staff. 
Consequently, there is an increased pressure on corporate education to demonstrate a 
return on the investments of their activities. 7 Hackett speaks of a “ measurement renaissance " 
for the human resource and executive education functions (1997, p. 5). 




Despite the renewed focus on evaluation, practitioners and academics have been largely 
unsuccessful in implementing evaluation instruments in practice (compare Smith 1993; 
Boverie et al. 1994; Gulpen 1996; Gotz 1998; Bramley 1999; Haring 2001 ). 8 The literature 
provides a wealth of theoretical conceptions of possible evaluation approaches, but many of 
these lack practical applicability and have therefore not been tested in real life. Nork (1991) 
provides an extensive list of causes for this evaluation dilemma. 

7 For example, 93 per cent of all training professionals at the 1996 ASTD International Conference stated that 
they are facing increased pressures to demonstrate return-on-investment results (Bassi & McMurrer 1999, p .1). 

8 The existing approaches and instruments are discussed in more detail in chapter 2. 
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Missing awareness towards evaluation 

- Costs of evaluation 

- Belief in the sure effectiveness of training and development activities 

- Fear of the people involved in training and development 

- Lack of appropriate concepts and instruments 

- Problem of attributing causes and effects " 

(Nork 1991, pp. 44-49) 

With our thesis we aim to advance the discussion on the evaluation of executive education. 
We address the common difficulties of practical evaluation through our research with a new 
evaluation tool. 

Including the field of evaluation, we have thereby presented briefly the four fields of interest 
in our thesis. We can therefore formulate our specific research objectives. 



1 .2 The Research Objectives 

" Evaluation ! There appears to be a lot of talk, some thought and precious little 
action. " 



(Hull 1959, p. 38) 

Over 40 years later, Hull’s remark remains valid to a large extent. Academics and 
practitioners alike are still searching for appropriate ways to evaluate executive education. 
Consequently, we derive two types of research objectives from the state of evaluation 
research. 



1.2.1 The Conceptional Research Objective 

First of all, we aim at developing a new evaluation approach, which is suited to the field of 
executive education. Thereby, we rely on existing approaches in the evaluation literature. 
Additionally, we attempt to use new ideas from other academic disciplines as well. Our 
underlying objective is to propose a new, and perhaps better, evaluation tool, which 
represents the conceptional contribution of our thesis to academia. 
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1.2.2 The Empirical Research Objective 



We acknowledge that there have been many theoretical evaluation tools in the past. Most of 
these ideas have never been implemented and consequently we know very little about their 
practical usefulness. Therefore, we choose to broaden our research objective and include a 
practical application and testing of our evaluation approach. Thereby, we aim at providing 
first hand experiences of the handling and the results of our approach. 

Given the unique research opportunity at the DaimlerChrysler Corporate University, we 
attempt to apply our new approach to the customised face-to-face programmes of the DCU 
(see chapter 4 for a detailed description of DCU). Our empirical research objective is thus to 
test our approach and to investigate the value of customised executive education. 
These executive programmes at the DCU have a strong strategic character (see quote in 
section 1.1.3) and therefore we include an additional focus on the development of strategy 
process capability (see subsection 2.2. 1.2). The real-life application of our approach to the 
aspect of customisation represents the empirical contribution of our thesis to academia. 

Overall, both the results of conceptional as well as the empirical work of our thesis should be 
of value to academics and to practitioners. We attempt to enhance the general 
understanding of the dynamics of our area of research, i.e. the new phenomenon of 
corporate university in Germany and the significance of customisation in executive 
education. We hope to enable new insights to the research area and to provide new 
solutions to the question of evaluation. 



1.3 The Research Assumptions and Hypotheses 

“Strategic Human Resource Management does not only serve to support the 
implementation of existing strategies, but should also be used to develop 
strategic capabilities. ... Learning plays a key role in the strategy making process, 
if not the most important role in the development of all new strategies.” 

(Kammel 2000, p. 210) 

Given the two types of research objectives, our research bases on several research 
hypotheses. These hypotheses represent the point of departure for our study. They have 
guided our conceptional work and our empirical research. 

First of all, we formulate two broad assumptions for our conceptional work. 
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• Existing evaluation research leaves room for improvement, i.e. there is a need for 
new evaluation approaches. 

• The value of executive education can be made visible though our new evaluation 
approach. 

Secondly, we base our empirical research on two specific hypotheses. 

• Customising executive education programmes is of value for an organisation. 

• Customised programmes have a higher potential to build strategy process 
capability than non-customised programmes. 

Our research objectives and hypotheses have led us to devise the following structure of our 

thesis. 



1.4 The Research Structure 

“Learning how to think evaluatively is learning how to learn. 



(Patton 1998, p. 226) 

The thesis represents a combination of conceptional work and empirical research. Thereby, 
we pursue a logical-deductive scientific approach (Ladriere 1984; Chalmers 1999) in the first, 
conceptional half (chapters 2 and 3). Subsequently, in the second, empirical half we follow 
an empirical-inductive approach (chapters 4 and 5). Overall, we emphasise an hermeneutic 
type of logic. We consider the evaluation of management development programmes similar 
to the construction of a puzzle. The complex and abstract learning of participants renders it 
difficult to prove any hypotheses in the non-experimental environment of the DCU. Instead, 
we collect pieces of evidence, i.e. the puzzle pieces, that assist us in better understanding 
the value creation of executive education. In terms of our research conception, we have 
devised the following structure. 

Chapter 1 introduces the area of research. We have taken a look at the research context , our 
objectives and hypotheses and are currently presenting the research structure. The purpose 
of chapter 1 is to introduce the reader to the field of research and explain the specificities of 
our thesis. 
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THEORY 



Chapter 1 Introduction and Overview 

Chapter 2 Theoretical Frame 

• Developing a flexible model of strategy process capability 

• Reviewing evaluation approaches in the Anglo-Saxon and German-speaking 
literature in psychology & pedagogy and finance & controlling 



Creating the theoretical foundation and providing input for the 
development of the new evaluation approach 



Chapter 3 New Evaluation Approach 

• The Strategic Tool for Evaluating Educational Returns (STEER) 
Development 
Description 
Implementation 



Providing the theoretical concept in order to enable the 
implementation in practice 



Chapter 4 Application in the Field 

• Description of the field of research - The DCU 

• Comparative study - Customised vs. non-customised executive development 
programmes 



Applying STEER to a real life situation and generating empirical data 



PRACTICE 



Chapter 5 Results 

• Dimension one: Results of the Comparative Study 

• Dimension two: Inferences on the quality of the tool 

Chapter 6 Limitations of the Research 
Chapter 7 Conclusions 



Figure 1 .5: The Structure of the Thesis 



Chapter 2 investigates the existing research on the two foci of our thesis. The first part of the 
Theoretical Frame takes a look at executive education. Thereby, we deal with the aspect of 
teaching strategy. We discuss the existing theory and develop our own definition of strategy 
process capability, which later serves as an important guidepost in the empirical research. 
Also we take a look at the roles of SHRD and at customising executive education. 
Subsequently, we review 12 evaluation approaches in the literature. We distinguish between 
the Anglo-Saxon literature and the German-speaking literature. Furthermore, we compare 
the existing evaluation approaches in pedagogy & psychology and finance & controlling. The 
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purpose of chapter 2 is to integrate the aspect of building strategy process capability through 
executive education and the question of evaluating executive education. Chapter 2 thus 
creates the theoretical frame for the conceptional work of chapter 3 and the empirical study 
of chapter 4. 

Chapter 3 illustrates the development of a new evaluation approach - the Strategic Tool for 
Evaluating Educational Returns. In the development of STEER we review the requirements 
of the approach with the help of the programme evaluation standards first (Sanders 1994). 
Subsequently, we discuss the conceptional sources of the development of our approach - 
Kaplan and Norton’s Balanced Scorecard (1992) and Kirkpatrick’s Four Level Evaluation 
Model (1979). In the description, we outline the structure of STEER and illustrate its 
rationale, the terminology, its evaluation instruments and the expected results. Afterwards, 
we propose ways to implement the approach. Finally, we consider the expected benefits of 
STEER. The purpose of chapter 3 is to present our new approach to the reader and provide 
insight to its development and future implementation. The chapter acts as an instruction 
manual for the evaluation in chapter 4. 

Chapter 4 outlines the empirical research of our thesis. We start with a description of the field 
of research - the DaimlerChrysler Corporate University (DCU) in the year 2000 - based on 
our explorative research of interviews, observation and document analysis. Subsequently, 
we describe the implementation of our new evaluation approach. We conduct a comparative 
study of internal DCU executive programmes and external public programmes of business 
schools by applying STEER. The objective of the comparative study is to test our evaluation 
approach in real-life. Thereby, we investigate the value creation of customising the internal 
executive programmes in comparison to the non-customised external programmes. 
Furthermore, we evaluate the potential of the two programme types with respect to 
developing individual strategy process capability. The purpose of chapter 4 is to outline the 
specificities of the field of research and to describe the empirical research that we have 
conducted in the framework of our thesis. 

Chapter 5 comprises the results of the comparative study. Here, we distinguish between 
dimension one results and dimension two results. Dimension one stands for the results of the 
comparative study generated by the application of STEER. These findings illustrate the 
differences and similarities of customised and non-customised programmes. Dimension two 
results provide a quality assessment of the approach itself. The application of the newly 
developed evaluation method enables us to make inferences on its usability. The purpose of 
chapter 5 is to guide the reader through the empirical findings of our thesis. 
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Chapter 6 puts the findings of chapter 5 in perspective. We discuss possible methodical 
shortcomings of the research and consider the transferability of our results. The purpose of 
chapter 6 is to assist the reader in judging the general significance of our findings. 

Chapter 7 closes our thesis. We summarise our results and draw possible inferences on their 
implications for the evaluation theory and practice. Furthermore, we make suggestions for 
further research on the grounds of our work. The purpose of the last chapter is to tie the 
pieces of our thesis together in order to reach an overall conclusion and to initiate further 
work in the field. 



16 




2 Theoretical Frame 



“Organisations should (...) pay more attention to systematically identifying their 
own specific developmental objectives and evaluating the outcomes of their 
developmental programmes. Too much of existing management development 
seems to be exclusively concerned with techniques and methods of the ‘train and 
hope ' variety. ” 



(Wexley and Baldwin 1986, p. 288) 

In chapter 1 we have introduced the overall context, objectives and hypotheses of our thesis. 
In the current chapter we provide now the theoretical frame of the two specified areas of 
research. According to our research objective, two streams of thought are presented here. 

In part 2.1 we formulate the research questions for the Theoretical Frame. These questions 
serve us as a broad guideline in the discussion of the literature. 

Given the newly emphasised strategic dimension of executive education we take a closer 
look at the meaning of strategic in the context of management development. In part 2.2 we 
develop a new definition of important individual capabilities, that are beneficial for the making 
and implementation of strategy, i.e. strategy process capability. Furthermore, we review 
ways of building strategy process capability through Strategic Human Resource 
Development. The aspect of customising executive education with respect to 
corporate/strategic issues is the another focus of this part. 

In part 2.3, we discuss 12 different evaluation approaches in the literature. Thereby, we 
distinguish psychological/pedagogical approaches and financial/controlling approaches on 
the one hand, and Anglo-Saxon contributions and German-speaking contributions on the 
other hand. Our review should enable the reader to develop a better understanding of what 
has been done in the area of evaluating executive education. 

Finally, in part 2.4 we show briefly how we use and integrate two streams of thought in 
terms of developing and implementing a new evaluation approach. 
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2.1 Research Questions for the Theoretical Frame 



“Nevertheless, the evaluation of the economic benefit from educational 
programmes is considered difficult in practice until today. ” 

(Haring 2001, p. 72; translated from German) 

In the Theoretical Frame we attempt to answer a selection of questions, which are relevant 
for the later development and application of our own evaluation tool in the field. The following 
broad research questions provide some orientation for mastering the literature review in 
theoretical chapter. 

First of all, we investigate the content of executive education programmes in more detail. 

LSI : What is meant by strategy process capability in the framework of teaching strategy to 

executives? 

LS2: How can Strategic Human Resource Development help to build strategy process 

capability? 

LS3: What does customisation entail in the area of executive education? 

In the second part of our Theoretical Frame, we investigate the method of our research. 

LS4: What theoretical and practical approaches exist to evaluate training and 

development? 

LS5: What are specific efforts to assess the impact - the return-on-education - of 

executive education? 

LS6: What are current problems and limitations in evaluation research? 

2.2 On Executive Education: The Content Foundation of Our Research 

“Corporate universities provide education, training and human resource 
development within companies, allowing these corporations to tailor programmes 
to specific business needs. ” 

(Anonymous, FT Survey on Executive Education 2001, p. 1) 
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Our research focus has been defined by the intersection of three research fields, namely that 
of Human Resource Development (HRD), corporate universities, and corporate strategy (cf. 
part 1.1). The focus of our research rests therefore on executive education in the framework 
of Strategic Human Resource Development (SHRD) and corporate universities. 

Specifically, we are interested in two aspects of executive education, which serve as the 
content foundation for our later empirical research. Thereby, we acknowledge that the field 
of executive education is not confined to these two aspects, but offers many more 
dimensions. 9 

First of all, in section 2.2.1 we investigate one of the most common objectives of executive 
education - teaching strategy. Strategic thinking as a core competence has received 
increasing recognition and is regarded as a principle source of sustaining competitive 
advantage (Prince & Steward 2000). Executive education is seen as one mean (among 
others), which can assist in developing the ability of executives to think strategically. Hence, 
we aim to provide a common understanding of strategic thinking, which we define as 
strategy process capability, in order to enhance the comprehension of our empirical 
research. Furthermore, we look at SHRD, which represents one way of building strategy 
process capability. Understanding SHRD assists us in specifying the characteristics of the 
executive education, which we investigate in our research. 

In section 2.2.2 we discuss the aspect of customising executive education. Customised 
executive programmes represent the object our research in the framework of a corporate 
university (cf. chapter 4). Customisation emerges as a new research area, with little existing 
theoretical or empirical research. We review the few sources found and describe the current 
state of research on customisation. Furthermore, we formulate our own assumptions of the 
value of customising executive education. 

The aspect of teaching strategy process capability and the aspect of customisation represent 
the two content foci in our empirical research later. A clear understanding of the two aspects 
is important to assist the reader in following the application of our evaluation tool. Overall, 
part 2.2 on executive education aims at answering the first three research questions (LSI to 
LS3). 



9 Garavan etal. (1999b) show the spectrum of executive education comprising not only face-to-face programmes, 
but also other forms of management development. 
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2.2.1 The Aspect of Teaching Strategy 



“It is important to note at the outset that it is individuals who think strategically, 
not organisations. ” 



(Liedtka 1998b, p. 120) 

The topic of strategic management can be found in almost all executive programmes. In 
terms of DCU’s executive education strategy represents the core objective. Therefore, we 
aim at developing an understanding of what is meant by teaching strategy to executives in 
the following subsection. 

In subsection 2. 2. 1.1 we derive a definition of strategy process capability. This definition is an 
important reference point in our analysis of how this capability can be developed through 
executive education. We show that different strategy process capabilities exist for different 
management levels in an organisation. Thereby, we formulate an own definition of strategy 
process capability and fill it with meaning for different management levels. At the end of the 
subsection, the first research question LSI should be answered: What is meant by strategy 
process capability in the framework of teaching strategy to executives? 

In subsection 2. 2. 1.2 we investigate the scope and goals of Strategic Human Resource 
Development (SHRD). The aspect to teaching strategy is partly rooted in a new 
understanding of training and development, which is depicted as SHRD (Hussey 1985; 
Garavan 1991; McClelland 1995; Garavan et al. 1995; Huselid et al. 1997; Burack et al. 
1998; Horwitz 1999; Schuler & Jackson 1999; McCracken & Wallace 2000). Hence, we 
outline the characteristics of SHRD and give the reader a better understanding of How can 
Strategic Human Resource Development help to build strategy process capability? (LS2). 



2.2.1. 1 Defining Strategy Process Capability 

“There is a growing recognition by senior management that in order to implement 
their plans, they require staff educated to think and act independently and with 
the ability to apply the latest management thinking. " 

(Prince & Steward 2000, p. 209) 
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The term strategy is a multifaceted concept, which is difficult to define universally (Mintzberg 
& Quinn 1991). However, we can broadly distinguish between the content of strategy and the 
process of strategy (Kammel 2000). 

The content of (corporate) strategy usually varies from firm to firm and depends on a 
complex array of different firm-specific, situational and environmental factors (Pettigrew & 
Whipp 1991; Mintzberg et at. 1998). Thereby, (corporate) strategy can entail different forms 
and purposes. 10 However, the content of strategy is of less interest in the framework of our 
thesis. We are more interested in the way the content of strategy is developed and 
implemented, i.e. the process of strategy. 

The process of strategy encompasses the making and implementation of strategy in an 
organisation. Hence, strategy process research investigates 

“How strategic processes work, which hidden capabilities come thereby into 
effect and what type of problems do executives have to expect?" 

(Kammel 2000, p. 161; translated from German). 

The literature provides many different ways of how strategy can be (descriptive) and should 
be (normative) made and implemented (cf. Chandler 1962; Quinn 1978; Ansoff 1987; 
Andrews 1987; Kay 1993; Mintzberg et al. 1998; Kammel 2000). 

“The need for strategic thinking has never been greater. ... [However,] there is no 
agreement in the literature on what strategic thinking is. " 



(Bonn 2001, p. 63) 

Given our framework of teaching strategy , we are interested in the art of making and 
implementing strategy and how this art can be taught to executives (Liedtka 1998b). The 
term “strategic thinking" is often used to describe the art of making strategy. Yet, it has been 
applied so widely that no common definition exists (Bonn 2001). 

Furthermore, the art of implementing strategy is often overlooked (Kay 1993). Therefore, we 
attempt to develop a new definition of strategy process capability, which comprises both 
the making of strategy, i.e. strategic thinking, and the implementation of strategy. 



10 For example, Mintzberg (1987a) provides an overview of the possible forms of the content of strategy. In his 
view strategy can be a “ plan ", a “ ploy ”, a “ pattern ”, a “position” and/or a “perspective” (pp. 12-16). 
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The importance of strategy process capability can be derived from the common problems in 
making and implementing strategy. Van Zwieten (1999) describes potential difficulties in the 
process of making and implementing strategy at different organisational levels. 

(1) “A new strategy is not being implemented by senior managers. ” 

(2) “Managers several layers down don’t understand the new strategy. ” 

(3) “Behaviour of managers isn’t congruent with new strategic initiatives they’ve 
helped create. ’’ 

(4) “The implementation plans for new business opportunity are unclear." 

(5) “Divisions are working at cross purposes. ’’ 

(6) “People in a department are unable to create strategic goals for themselves. ’’ 

(Van Zwieten 1999, pp. 49-51) 

Van Zwieten demonstrates that successful strategy-making and implementation requires 
middle and lower managers to possess sufficient strategy process capability (Kay 1993; 
Hayes et al. 2000). 

In developing a new definition of strategy process capability, we take four steps. First of all, 
we distinguish between organisational and individual capability. Then we show that 
specific capabilities can be defined for four generic hierarchical levels in an organisation. 
Thirdly, we define capability as consisting of skills, knowledge and mentality. Finally, we fill 
the definition with content for four hierarchical levels. 

Step One - Distinguishing Between Organisational and Individual Capability 

The strategy literature shows that organisations can take very different approaches to the 
process of making and implementing strategy. These approaches vary from systematic 
planning to more reactive and ad hoc forms of strategy process (Chandler 1962; Andrews 
1987; Ansoff 1987). Furthermore, the process itself can be a designed top-down procedure 
or an emergent bottom-up approach (Quinn 1978; Kay 1993; Mintzberg et al. 1998). 11 



11 Mintzberg etal. 1998 provide a useful compendium often different approaches of strategy making and 
implementation. 
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Descriptors 


Command 


Symbolic 


Rational 


Transactive 


Generative 


Style 


Imperial 

Strategy driven 
by leader or 
small top team 


Cultural 

Strategy driven 
by mission and 
vision of the 
future 


Analytical 

Strategy driven 
by formal 
structure and 
planning system 


Procedural 

Strategy driven 
by internal 
process and 
mutual 
adjustment 


Organic 

Strategy driven 
by organisational 
actors’ initiative 


Role of top 


Commander 


Coach 


Boss 


Facilitator 


Sponsor 


management 


Provide direction 


Motivate and 
inspire 


Evaluate and 
control 


Empower and 
enable 


Endorse and 
sponsor 


Role of 


Soldier 


Player 


Subordinate 


Participant 


Entrepreneur 


organisational 

members 


Obey orders 


Respond to 
challenge 


Follow the 
system 


Learn and 
improve 


Experiment and 
take risks 



Table 2.1 : An Integrated Framework of Strategy-Making Processes 

Source: Hart & Banbury 1994, p. 254 

Given the manifold forms of strategy process Hart and Banbury (1994) emphasise the 
importance of possessing an organisational strategy process capability in order to be 
able to manage the making and implementation of strategy successfully. 

“Firms able to accumulate several modes of strategy-making process ‘capability’ 
appear to outperform less process capable organisations. ... [Specifically] Firms 
demonstrating high strategy process capability show high levels of performance. ” 

(Hart & Banbury 1994, p. 265) 

The underlying idea is that an organisation possesses the capability to perform different 
forms of strategy making and implementation. For example, in their framework Hart and 
Banbury list five styles of making and implementing strategy. Thereby, strategy process 
capability implies that an organisation can handle more than one style successfully. 12 

Unfolding organisational strategy process capability, we discern the importance of individual 
strategy process capabilities (Bonn 2001). As we can see in Hart and Banbury’s framework, 
the organisational capability consists of the sum and the combination of the capabilities of 
executives and employees (Schwuchow 1992). 



12 The need for handling more than on style can arise from changes in style of time or from the heterogeneity of a 
large organisation. 
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“The core competencies of individuals are thoughtfully linked to the core 
competencies of the organisation." 



(Burack etat. 1998, p. 17) 

Hence, we can assume that there is a set of general individual strategy-making and 
implementation capabilities that supports multiple types of strategy processes and therefore 
increases the overall organisational strategy process capability. 

Before proceeding to step two, we emphasise the difference between ‘strategic capabilities' 
in the literature and strategy process capability in order to ensure more clarity in the 
development of our definition. Our research deals with the latter and we distinguish the two 
types very clearly. 

“Strategic capabilities" or “competencies" form the basis of the resource-based theory of 
strategy (Dierickx & Cool 1989; Barney 1991; Hamel & Prahalad 1994; Collis & Montgomery 
1995). The underlying idea of the resource-based theory is that firms should identify and 
develop their strategic capabilities and resources, often named core competencies, in order 
to build a competitive advantage. For example, in the case of DaimlerChrysler one strategic 
capability certainly is its engineering competence. In line with the resource-base view, 
DaimlerChrysler should therefore concentrate on enhancing its engineering competence to 
build a sustainable competitive advantage. 

The concept of strategy process capability addresses the proficiency of making and 
implementing strategy (Bourgeois & Brodwin 1984; Alexander 1985; Teece et al. 1997). 
Many authors use the term “strategic thinking" (Mintzberg 1994; Christensen 1997; Liedtka 
1998b; Bonn 2001) to describe the capability of making strategy. However, we consider it 
important to integrate the making and the implementation of strategy into one concept of 
capability. Most strategies fail because they are not effectively implemented (Kay 1993). 
Therefore, we include both elements in our definition. 

Step Two - Defining Four Generic Hierarchical Levels 

Specifying individual strategy process capability we can see that different layers in the 
organisational hierarchy play different roles in the strategy process. Thereby, we assume 
that strategy process capabilities can be found at different levels of an organisation. 
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Hart and Banbury (1994) distinguish between top management and organisational members 
in their framework. Their dichotomy can be extended to a finer distinction of organisational 
levels. Burack et ai. (1998) proposes three generic organisational levels in their analysis. 
They include a middle management level and thus distinguish between all employees , mid- 
level managers and senior managers. Further, von Rosenstiel (1993) suggests to separate 
middle management from lower management. We discern a finer distinction than the three 
approaches - distinguishing four different organisational levels (cf. Munch 1996). We define 
the top management as the few decision-makers at the top of an organisation (the group of 
leading executives that Mintzberg (1979) calls the Strategic Apex). Furthermore, we 
distinguish between middle management and lower management 13 (cf. Andrews 1987; 
Mintzberg regards these two levels as Middle Management). The fourth level is called 
employees for simplification reasons (Mintzberg’s Operating Core). 



Hart and Banbury (1994) 


Burack eta/. (1998) 


Von Rosenstiel (1993) 


Our Distinction 


Top management 


Senior managers 


Top management 


Top management 


Organisational members 


Mid-level managers 


Middle management 


Middle management 


Lower management 


Lower management 


All employees 




Employees 



Table 2.2: Distinguishing Organisational Levels 

Source Hart & Banbury 1994; Burack et al. (1998); von Rosenstiel (1993) 



The rationale for dividing an organisation into four generic levels is to facilitate the analysis of 
individual strategy process capability. The distinction assists us in evaluating the contribution 
of executive education as most of the target group in our research consists of members of 
middle management. Thereby, we assume that the different levels require different strategy 
process capabilities. We explain the differences by the responsibilities of the respective 
actors on each organisational level (Wimmer 1995). Von Rosenstiel (1993) portrays the 
relationship between strategic decision-making and implementation for different management 
layers. These layers can be directly translated into the three organisational actor groups 
mentioned above - the top, middle and lower management. 



13 An illustration might assist in making the distinction. Middle management could include divisional leaders, 
country representatives, heads of subsidiaries and plant directors. Lower management could mean managers 
heading smaller departments and teams. 
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Being aware of the generalisations implied, von Rosenstiel’s descriptive illustration 
demonstrates the mixture of work activities and responsibilities for each organisational 
group/level. According to von Rosenstiel, the top management spends most of its time on 
making strategic decisions and planning decisions. Further, the middle management deals 
with both the making and the implementation of strategy. Finally, the lower management 
spends almost all of its time on the implementation. Our definition follows von Rosenstiel’s 
classifications and defines the individual elements of strategy process capability for each 
hierarchical level. Therefore, we assume that the members of the four organisational levels 
play different roles according to the prevailing strategy process type in the organisation. 
Hence, there are significant differences in the required strategy process capability with 
respect to each group of organisational actors. 

Step 3 - Distinguishing Between Skills, Knowledge and Mentality 

The next step in the definition of strategy process capability entails the definition of capability. 
In the framework of our thesis, we define strategy process capability through the following 
assumption. Strategy process capability consists of strategy skills , strategy knowledge and 
strategy mentality. 

In line with the literature, we have separated capability into skills, knowledge and mentality 
(cf. de Bettingies 1975; Hentze 1986; Schwuchow 1993; Schein 1993; Brockbank 1999; 
Hayes et al 2000). Combined, the skills, knowledge and mentality form the strategy process 
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capability of an individual. The term ‘strategy’ in this content means that the skills, knowledge 
and mentality are related to the making and implementation of corporate strategy (table 2.3). 



Strategy Process Capability 



Strategy Skills 




Strategy Knowledge 




Strategy Mentality 


For example: 




For example: 




For example: 


Analysing and communicating 




Knowing the content of 




Possessing willingness and 






corporate strategy 




drive to change things 



Table 2.3: The Three Elements of Strategy Process Capability 



We assume that the possession of the ‘right’ skills, the appropriate knowledge and ‘the’ right 
mentality provides a person with a positive strategy process capability. This implies that the 
person is able to work well in different types of strategy processes and can handle different 
contents of strategy effectively. Overall, we assume that the possession of these individual 
elements of strategy process capability is beneficial for successful strategy-making and 
implementation in an organisation. 



We acknowledge that in our discussion we take an isolated look at strategy process 
capability. Thereby, we agree, that in reality, strategy process capability is likely to blend into 
many other capabilities, as for example financial or inter-personal skills. Our definition is not 
meant to reflect reality in a minute way, but to enhance our understanding of each 
hierarchy’s distinctive principal skills, knowledge and mentality. We are aware of the danger 
of becoming too generic, but we chose our differentiation in order to highlight the ideal 
distinctions. 



Step Four - Filling Strategy Process Capability With Content 

In the final step of our definition we fill the three elements of strategy process capability with 
content, i.e. we propose which skills, what knowledge and what kind of mentality are part of 
an ideal strategy process capability for each hierarchical level. In terms of defining the 
content of strategy process capability we have used several sources in the literature. These 
sources have provided concrete skills, types of knowledge and descriptions of mentality, 
which we have included in our definition subsequently. We have thus tried to merge the 
following ideas from the literature, which we found most relevant. 

Burack et al. (1998) offer a short list of core competencies for three hierarchical levels, which 
include strategic competencies as well (see appendix 8.2). Burack et al. confirm that senior 
managers, i.e. top management according to our terminology, should initiate strategy and 
lead change. The most interesting insight from their work however is that mid-level 
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managers, i.e. the middle management, should implement the change and think strategically 
as well. Thereby, Burack et al. allow for a top-down and a bottom-up strategy process. 

Bonn (2001) offers three elements of strategic thinking, which serve us in the definition of 
strategy process capability as well (appendix 8.3). She starts with a holistic understanding of 
the organisation and its environment (strategy knowledge). Then she names creativity as an 
important strategy skill. Finally, she mentions vision as an element of strategy mentality. 

Liedtka (1998b) devises a definition of strategic thinking as well (appendix 8.4). Her elements 
of strategy thinking are “having a systems perspective ’’, “being intent focused", “thinking in 
time”, “being hypothesis-driven” and “possessing an intelligent opportunism" (cf. p. 122, 
1998b). 

Fauth (1992) provides a detailed description of strategy skills needed by executives. Under 
the heading of “Entrepreneurial Competence” (p. 233-235) he lists skills of “Strategic thinking 
and planning" (p. 234). Thereby, Fauth emphasises the importance of analytical skills, 
communication skills, planning and realisation skills (cf. appendix 8.5). Furthermore, Fauth 
refers to strategy knowledge, which he defines as knowledge of corporate goals, the market 
and the competitors. 

Porter (1996) depicts the role of leadership in the process of making and implementing 
strategy. Thereby, he requires executives to be able to communicate and teach strategy. 

“General management is more than stewardship of individual functions. Its core is 
strategy: defining and communicating the company’s unique position, making 
trade-offs, and forging fit among activities. ... One of the leader’s jobs is to teach 
others in the organisation about strategy - and to say no. ” 



(Porter 1996, p. 77) 

We integrate the leadership standards of DaimlerChrysler in our definition. The so called 
“LEAD criteria” include the aspect of strategy thinking and establishing direction which we 
have used as a source as well. 

Larreche (1999) stresses the importance of decision-making and building commitment as 
part of strategy process capability (appendix 8.6). 

In the following table (table 2.4 on the next page) we formulate an ideal framework of 
strategy process capability. The principle elements of strategy process capability are found 
on the left of the table. Additionally, we define exemplary activities for each organisational 
level. We understand our definition not as a complete list. However, we are confident that our 
definition enables a good understanding of strategy process capability. 
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Table 2.4: Defining Strategy Process Capability for Four Groups in a Simplified Organisational Hierarchy 
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The purpose of the table is to provide a guideline for the design, implementation and, above 
all, for the evaluation of strategic management development programmes. By defining 
strategy process capability as skills, knowledge and mentality, the specific objectives of 
building strategy process capability through executive education can be laid out. 

Summarising section 2.2.1. - we proposed a new definition in the framework of the strategy- 
making process. We understand strategy process capability as the ‘basic tool kit’ for making 
and implementing strategy more effectively. Regardless of the type of strategy process, we 
assume that the possession of strategy process capability is beneficial to the success of an 
organisation. Four different actor groups were derived and defined. Leaving room for the 
different types of strategy process, our definition discerned core elements of strategy process 
capability, i.e. strategy skills, knowledge and mentality, which were subsequently defined for 
all types. The section has thereby answered our first research question LSI, i.e. “What is 
meant by strategy process capability in the framework of teaching strategy to executives?" 

In the following subsection we take a look at ways of building and developing strategy 
process capability. 



2.2.1. 2 Defining SHRD as a Way to Build Strategy Process Capability 

“Management development could make a contribution to the field of strategic 
management by focusing on how to address the practical problems of integrating 
competencies in such a way that new competencies can develop earlier, faster 
and better, while such competencies remain at the same time subject to inquiry, 
reframing and reshaping. ” 



(Overmeer 1997, p. 258) 

Referring back to the introduction of our thesis, in the following subsection we provide the 
connecting piece between strategy process capability and the role of corporate universities 
as providing strategic executive education. The concept of the corporate university is partly 
rooted in a new understanding of training and development, which is depicted as Strategic 
Human Resource Development (SHRD) (Hussey 1985; Garavan 1991; McClelland 1995; 
Garavan et al. 1995; Huselid et al. 1997; Burack et al. 1998; Horwitz 1999; Schuler & 
Jackson 1999; McCracken & Wallace 2000). Hence, we define the characteristics of SHRD 
in the following subsection. Thereby, we give the reader a better understanding of how 
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management training and development can add value to the strategy process capability. As 
a result, we answer the second research question: How can Strategic Human Resource 
Development help to build strategy process capability? 

The use of the term strategic might be misleading and therefore we take a brief look at a 
definition. Garavan (1991) formulates one of the most widely accepted definitions of SHRD. 

“[SHRD] is best seen as the strategic management of training , development and 
of management/professional education interventions, so as to achieve the 
objectives of the organisation while at the same time ensuring the full utilisation of 
the knowledge in detail and skills of individual employees. It is concerned with the 
management of employee learning for the long term keeping in mind the explicit 
corporate and business strategies . " 



(Garavan 1991, p. 17) 



The definition shows, how SHRD emphasises the link between learning and corporate 
strategy. According to SHRD the development of personnel should be aligned to the goals 
and objectives of the organisation. In table 2.6 we can see the underlying logic, how training 
could be more aligned and integrated into the strategy process. 



t 



1 . No systematic training 

2. Isolated tactical training 

3. Training integrated with operational management 

4. Training is the means for implementing corporate strategy and achieving change 

5. Training and learning possibilities help to shape strategy 

6. Training and learning are processes through which strategy is formulated 



Table 2.5: The Scale of the Strategic Maturity of Training 

Derived from Lee 1996, p. 7 



The literature thereby regards Human Resource Development as offering a more long-term 
advantage through distinctive human capital (cf. Olian et al. 1998), which includes strategy 
process capability as one important competency as well (cf. Brockbank 1999). 



In line with Olian et al. and Brockbank’s argument, we identify three roles of SHRD in the 
framework of our research, i.e. the informing, the aligning and the advancing role. Through 
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each role SHRD contributes to the making and implementation of (corporate) strategy in a 
different way. 



Informing Role 


Aligning Role 


Advancing Role 


SHRD provides information to the 
high-level decision-makers on the 
skills and knowledge required for 
the implementation of a given 
strategy 


SHRD develops and aligns the 
skills and knowledge required for 
the implementation of a given 
strategy through training and 
development 


SHRD improves and advances 
the strategy process capability of 
the high-level decision-makers 
through training and development 



Table 2.6: The Three Roles of Strategic Human Resource Development 



First, SHRD can integrate the human resource function in the strategy-making process 
(Garavan 1991; McCracken & Wallace 2000; for the specific characteristics of SHRD 
compare appendix 8.7). The close link of HRD and strategy then provides the information 
about which skills and what knowledge (i.e. capabilities/competencies) are required for the 
effective and successful implementation of corporate strategy (cf. McClelland 1995). In the 
framework of our thesis the first role is called informing role. For example, in the case of an 
external expansion strategy, HRD experts would point out beforehand that entering a new 
market could prove difficult due to middle management’s lack of language and culture 
capabilities. 

Secondly, as a consequence of the first one, SHRD can assist in implementing a specific 
strategy by building and enhancing the required skills and knowledge (cf. Barney 1995; 
Kamoche 1996). These skills and knowledge become strategic because they are essential 
for the implementation of the corporate strategy (cf. strategic capabilities in subsection 
2.2. 1.1). We apply the term aligning role of SHRD (cf. Garavan 1991; Garavan et al. 1995; 
Lee 1996; Olian et al. 1998; Noe 1998). Using the same example as above (the expansion 
strategy), the second role would imply that SHRD develops the language and culture skills of 
the middle management in order to support the implementation of corporate strategy. 

Thirdly, SHRD can improve the strategy-making process directly by building general strategy 
process capability (cf. section 2.2. 1.1), which is needed to make and implement corporate 
strategy (cf. Burack et al. 1998; Liedtka 1998b; Prince & Stuart 2000). We use the term 
advancing role. Referring again to the example above (expansion strategy), SHRD implies 
that the top and middle management are trained to be more capable of assessing and 
deciding to what extent the expansion strategy should be initiated and implemented. In other 
words, top and middle management are themselves more capable to develop and formulate 
the expansion strategy. 
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Hence, SHRD can develop the strategy process capability of high level decision-makers, 
which proves to be beneficial in the strategy process overall. Blaesi et al. (2000) emphasise 
the importance of the advancing role and thereby show its underlying logic. 

"Human Resource Development plays a central role with respect to the 
implementation of corporate strategy. Only those employees, who possess the 
competence and the understanding for entrepreneurial and environmental 
influences, are able to implement successfully a given strategy. ... Often, the 
strategy of a company is not badly drafted, but the human capability is missing to 
implement it. ... At the end of the day the implementation capability of well 
educated employees determines the success of implementing strategy. ” 

(Blaesi et al. 2000, pp. 24-25; translated from German) 

The SHRD literature offers a selection of training and development methods, which are 
suited to building strategy process capability. For example, Garavan et al. (1999b) provide an 
extensive compendium of methods of management training and development in the 
framework of SHRD. 14 



Method 


Description 


In-house development programmes 


Courses organised by internal or external trainers and conducted internally 


External development programmes 


Mangers sent to outside courses, for special training or academic degrees. 
Trainers are external to the organisation 


Task force/project groups 


Cross-departmental groups that study organisational problems and/or carry 
out special assignments 


Seminars/workshops 


Teach-each-other events for pooling expertise 


Action learning 


Experimental learning, based on the concept of the learning cycle and 
around the principles of action learning 


Outdoor management development 
programmes 


Programmes designed to develop such skills as leadership, self 
confidence, self awareness through a variety of experiences including 
outdoor physical challenges 



Table 2.7: Strategic Management Development: Alternative Strategies 

Source: Garavan etal. 1999b, p. 1999 (selection) 



14 Garavan et al. (1999b) list further activities in the compendium as well, as for example, Job Rotation. Table 2.7. 
provides a selection from the more extensive list by Garavan et al.. 
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The list demonstrates that SHRD draws on rather traditional forms of training and 
development. Most of the items on the list are well known in the field of HRM. Therefore, we 
can assume that the development methods are very similar to non-SHRD training and 
development. 

Taking a look at general teaching methods Burgoyne and Stuart (1991) provide a list that 
ranks training and development methods according to their frequency and use. The items are 
sub-elements to the different forms of management development referred to by Garavan et 
al.. With the help of Burgoyne and Stuart’s list, we can build a broader understanding of 
possible teaching methods used in SHRD. 

“( 1) Lecture 

(2) Games/simulations 

(3) Projects 

(4) Case studies 

(5) Experiential 

(6) Guided reading 

(7) Role playing 

(8) Seminar 

(9) Programmed instruction' 115 

(Burgoyne & Stuart 1991, p. 28) 

Their research shows that the most common form of executive education is the traditional 
lecture form, followed by more action oriented elements. However, one should keep in mind 
that the research provides a picture of the state-of-the-art ten years ago. More recently, new 
trends have emerged, for example Action Learning (cf. Harrison & Leitch 2000). New 
technologies have enabled more effective Distance Learning, which comprises Virtual and 
Technology Mediated forms of management training and development (cf. Bassi et al. 1998). 

Schwuchow (1992) devises practical case of how strategy process capability can be trained 
and developed. He describes how skills, knowledge and attitudes are developed in the 
framework of SHRD. We emphasise that he is not referring to the specific skills, knowledge 



15 The list is based on research based on the analysis of 15 management development programmes with respect 
to their methods (Burgoyne & Stuart 1991). 
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and mentality of our definition. Nevertheless, we can derive a broader comprehension of the 
different ways of building strategy process capability. 



Education Content 


Learning Process 


Teaching Method 


Skills 


Cognitive, affective 


Content- and process-oriented methods 






Case study, planning game, project work, 
exchange of experiences 


Knowledge 


Cognitive, intellectual 


Content-oriented methods 






Lecture, self-study 


Attitudes 


Affective, emotional 


Process-oriented methods 


(Mentality) 




Role play 



Table 2.8: Education Content and Teaching Methods 

Derived from Schwuchow 1992, p. 101 



The most important insight from Schwuchow’s approach is that the three elements of 
strategy process capability are developed through different learning processes. All of these 
learning processes are supported by teaching methods, which are found in common 
executive education programmes. We can see that our definition of strategy process 
capability can indeed serve as the foundation for evaluating the learning objectives of 
executive education. 

Closing the subsection we remark that the literature on SHRD remains not without criticism. 
The current state of SHRD presents a holistic and integrated perspective (cf. Garavan et al. 
1999b). However, the apparent theoretical uniformity of large parts of the research can be 
misleading. Despite the underlying definition of SHRD, there are principally two problems, 
which are worthwhile mentioning. 

The first problem is the common sense nature of the new concept of SHRD. In particular, the 
SHRD literature presents the informing and the aligning roles as new and emerging. 
Thereby, the literature suggests that training and development have not been very well 
aligned to strategy until the emergence of SHRD. Following the claims of SHRD, it seems 
that training and development had been done without any link to the strategy-making and 
implementation process. We doubt whether these claims can be considered true and 
justified. Instead, it appears likely that strategists have known that it was a guarantee for 
success to have the Tight people, with the right skills, at the right place in the right time’ (cf. 
Stiefel 1974; Burack 1986). Excellent leaders and strategists have always considered their 
human resources before drafting and implementing their strategies (cf. von Clausewitz 1832- 
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37 in Hallweg 1972). Most of them have invested in the strategic training and development of 
their employees before the emergence of SHRD. A common problem in the past could have 
been that top management often neglected human resource development and that personnel 
staff did not consider itself important for the implementation of strategy (cf. Schwuchow 
1992). But to claim that the strategic link of SHRD is new and revolutionary seems 
exaggerated to us. 

A second difficulty is the foremost theoretical emphasis of SHRD. The literature tends to 
regard SHRD as the panacea for the failures of so many strategies in the past. However, 
there is a considerable lack of empirical evidence to support this theory. Few firms have 
defined and implemented a high-level SHRD function as clearly as demanded in the 
literature (cf. Hackett 1997). There are several anecdotal pieces of evidence for the value of 
SHRD (Garavan et a\. 1995). However, the existing empirical material reflects a highly 
heterogeneous picture, contrasting the theoretical work (cf. McClelland 1995; Olian et al. 
1998; Becker & Huselid 1999). Therefore, it might be best to consider the practice of SHRD 
under a contingency theory perspective (cf. Garavan et al. 1999a). This implies that more 
and more firms emphasise the strategic relevance of training and development. Yet, the 
understanding of SHRD, its objectives and the identified processes differ extensively (cf. 
Garavan et al. 1999b) from firm to firm. It is therefore contingent on the specific 
characteristics of the firm and its environment. 

Despite these two areas of criticism, one can discern a new emphasis on the strategic side of 
HRD in the literature and in practice. The growing interest in the strategic role of corporate 
universities underlines the motivation to take a closer look at SHRD. Our research is 
primarily interested in the process and the impact of the third role of SHRD, namely the 
advancing role. Hence, we investigate how far executive education can build strategy 
process capability. 

Concluding the subsection - we addressed the second research question (LS2) “How can 
Strategic Human Resource Development help to build strategy process capability?’’. SHRD 
emphasises the link between corporate education and corporate strategy. It claims that the 
development of personnel should be aligned to the goals and objectives of the organisation. 
We perceive three general roles of SHRD. It can inform decision-makers on the stock of 
capabilities and it can develop these capabilities to support the implementation of strategy. 
Additionally, SHRD can build strategy process capability through its advancing role, i.e. by 
developing the strategy skills, knowledge and mentality of its high level decision-makers. 
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In the following section we take a look at the aspect of customising executive education in 
order to complete the content foundation of our research. 



2.2.2 The Aspect of Customisation 

"Customised training is being done around the idea of ‘strategic intent’ ... and 
stresses the overall strategic outlook ability. Business schools used to offer many 
types of open enrolment programmes where the information was just ‘nice to 
know’, but now emphasis is on developing a much more far-reaching set of skills, 
such as strategic planning. ” 



(Moore in Smith 1997, p. 2) 

The second content focus of our thesis rests on the aspect of customising executive 
programmes in the framework of a corporate university (Walton 1999). In our empirical part 
we apply our new evaluation approach to the comparison of customised executive 
programmes with non-customised programmes (cf. part 4.2). Therefore, we outline the scope 
and implications of customisation in the following subsections. Our goal is to enhance the 
understanding of the phenomenon. 

In subsection 2.2.2. 1 we discuss customisation as a new area of research in the field of 
executive education. We outline that customisation represents a new trend in management 
development. Furthermore, we show that the aspect of customisation remains underexposed 
in the literature. In terms of empirical research, we have not found any study done on the 
value of customisation in executive education. 

In subsection 2. 2. 2. 2 we discuss the underlying opinions of customisation in practice. We 
demonstrate that the value of customisation is a debated issue among practitioners. Given 
the novelty of the subject, we try to outline the existing positions towards customisation. 
Thereby we aim at providing a better understanding of the subject for our later empirical 
research. 
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2.2.2.1 Customisation: A New Field of Research 



“Companies are becoming more vocal customers of MBA [business] schools and 
they are calling the shots in terms of how many students they will commit and are 
now demanding customisation of programmes.” 



(Meister in Murray 2001, p. 2) 

The last ten years has seen a change in the relationship of corporations and business 
schools. Until the late 1980s corporate education and business schools “tended to maintain 
an arm’s length relationship” (Ghoshal et at. 1992, p. 65). In general, business schools 
offered two types of products in education. 

On the one hand, the MBA 16 represented the traditional form of business school education 
(Cheit 1985). Originated at the first American business school, the Wharton School at the 
University of Pennsylvania 17 , the MBA rose quickly to be the most popular post-graduate 
degree in business in North America. 

On the other hand, business schools started to offer shorter executive programmes, which 
addressed full-time managers. These executive programmes were several days or weeks 
long and were conducted by the respective business school on site. 

Traditionally, both types of courses, i.e. the MBA and the executive education, were 
developed and run by the business school independently. At the beginning of the 1990s, 
however, a new trend of executive education emerged. 

“There is increasing evidence that corporations are demanding a more direct 
linkage between management education and corporate development. This 
demand is reflected in the growing popularity of company-specific courses and, 
more threatening for business schools, in companies developing such 
programmes in-house, renting business school faculty as necessary. ” 

(Ghoshal et al. 1992, pp. 62-63) 



16 The Master of Business Administration is a postgraduate degree for students who possess some professional 
experience already. For example, many business schools require at least five years of professional experience 
from applicants. 

17 The Wharton School was founded in 1881 with the help of a donation from Joseph Wharton, a wealthy 
Philadelphia businessman (Cheit 1985). 
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Ghoshal et al. describe a new form of executive education. In line with new HRD 
developments, like for example corporate universities, corporations started to demand 
tailored or customised executive programmes, which were exclusively designed to their 
needs and their employees. 

" Companies have turned to management learning and development as a means 
to help achieve strategic objectives. ” 



(Baldwin & Danielson 1998, p. 2) 

The strategic objective of education, that Baldwin & Danielson (1998) describe, is regarded 
as one of the reasons for more customised education. 

“The growing importance of management education and the need for 
organisations to implement strategic initiatives has led to a growing demand for 
bespoke [customised] programmes. ” 



(Prince & Stewart 2000, p. 212) 

Today, these customised programmes are increasingly found in practice and are quickly 
growing in number (Stiefel 2000). These customised programmes differ from the more 
traditional approaches. 

“Much traditional development activity might be characterised as a ‘parade of 
stars’ training, in which a succession of external experts are retained to present 
their pre-packaged messages. Combining the focus on real work with a problem- 
then-theory model, management learning becomes less about listening to 
answers from experts and more about collecting valid information from a variety 
of sources to be considered in the local business context. The key distinction is 
that expertise is sought in response to a problem, not in advance of them. ” 

(Baldwin & Danielson 1998, p. 4) 

A second quote from Liedtka et al. (1999), one of the few contributions that we were able to 
identify in the literature, confirms the novelty of customisation. 

“[There is] a larger trend, evident in the field of executive education itself, towards 
more customised and focused programmes and away from the more traditional 
functionally organised, open enrolment programmes. Viewed from this 
perspective, executive education is seen as a strategic tool, which reflects the 
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urgent need for executive development to promote both individual learning and 
organisational adaptation and renewal 



(Liedtka et al. 1999, p. 405). 

Yet, until today, the literature has offered very few contributions on the aspect of 
customisation. Many sources come from business newspapers that describe the new trend 
(cf. the FT Survey on Executive Education 2001, Corporate University Review). From an 
academic perspective we were unable to identify sufficient empirical research on 
customisation (cf. Liedtka et al. 1999). 

Given the lack of definitions of customisation, we formulate our own definition of the 
aspect of customisation, which we refer to throughout our thesis. 

Customisation implies that the content, and to some extent the methods, of an executive 
education programme are adjusted to corporate needs, i.e. aligned to the strategic issues of 
the company for which the programme is customised. Thereby, the programme content and 
methods are developed through a co-operation of business school professors and corporate 
staff. 

The result of customisation is a programme 

• whose content and methods are tailored to the company, 

• which is exclusive to the executives of the company, and 

• which is given by the professors of the business school and internal corporate speakers. 

In line with the quote from Baldwin and Danielson (1998), we can see that customisation 
attempts to address and solve specific corporate problems. However, the literature does not 
specify the necessary degree of customisation. Furthermore, it omits the aspect of possible 
quality controls of customisation. 

Summarising subsection 2.2.2.1 - the aspect of customisation remains a vague 
phenomenon in the literature, with few concrete research and examples. We provided a first 
definition of customisation in order to structure the field. In practice, first opinions have 
formed with respect to customisation, which we review in the following subsection. 
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2. 2. 2. 2 Customisation: A Debated Area in Practice 



"While the potential for using executive education to forge a different kind of 
relationship between business and business schools has been recognised, there 
is not much evidence that this potential has actually been exploited to any 
significant extent . " 



(Ghoshal et al. 1992, p. 51) 

Customisation presents an important element in our thesis. The application of our evaluation 
approach STEER serves to assess the value of customised executive programmes at the 
DaimlerChrysler Corporate University. As we have seen above, little academic work has 
been found on the specific value of customisation. However, the literature reveals that 
customisation has become a debated phenomenon in practice. 

First of all, we can see that several corporations have started to design and run customised 
programmes. Ghoshal et al. (1992) describe a new type of partnership in executive 
education between INSEAD and Digital Equipment Corporation in the early 1990s. The 
design and implementation involved many issues related to Digital’s specific needs and 
concerns. One of the goals of the new programme was to create a linkage to corporate 
development. 

“The objective of [the programme] was not simply to facilitate implementation of a 
strategy developed by the top management, but to evolve a strategy and develop 
action plans for implementing that strategy. It has often been claimed that 
strategy formation is a learning process, and [the programme] was designed to 
create a direct linkage between learning among managers and action for 
strategic change within the company. ” 



(Ghoshal etal. 1992, p. 52) 

We can see that customisation possesses close ties to SHRD as it focuses on specifying the 
content of a programme to the corporate needs and strategy. Thereby, we can deduce that 
SHRD provides a useful theoretical framework to comprehend the underlying rationale of 
customisation. 

More recently, Liedtka et al. (1999) conducted a study on the sustainability of an executive 
education experience. Although the focus of their research rests on a different phenomenon 
of management development, Liedtka et al. mention a new partnership between a major 
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business school and a financial service organisation. Again, we find a similar objective of 
executive education. 

“The intent of the partnership was to develop and deliver a customised leadership 
programme for the upper-middle management of the organisation, focused on the 
topic of leading strategic change.” 



(Liedtka et al 1999, p. 405) 

The two examples show clear similarities to the type of programmes that we investigate in 
our empirical research (cf. chapter 4 and 5). Both include a new emphasis on teaching 
strategy and on customisation. 

However, both studies have different research agendas and thus do not investigate the 
value, i.e. the advantages and disadvantages, of customising executive education. 

We find that the value of customisation currently represents a debated aspect among 
practitioners. On the one hand, the advocates of customisation claim that a corporation 
benefits from the partnership with the business school. The close tie with an external 
academic institution ensures a direct exchange of the latest findings of business research. 
Participants are taught by renowned experts in the field, who can integrate the internal 
company perspective with the external trends of the business world The corporation can 
thereby leverage the teaching expertise and the brand name of the business schools in order 
to increase the value of its own educational activities. 

“[The customer] expects us to understand its business and its emerging 
strategies with case studies that are relevant to its business. ” 

(Communal on the role of business schools in Wood 2001 , p. 3). 

Communal states the new expectations of corporations towards business schools clearly. 
The arm’s length relationship seems to turn into a closer partnership, as Lorange describes it 
in the following quote. 

“The business school can provide various in-company programmes, or modules 
of larger programmes ...as true teaching partners. 

Second, the business school can apply its faculty and research staff to the 
development of particular [company specific] teaching material. 
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Third, the business school can provide professional resources in a mentoring 
capacity. ... The faculty resource can thus serve both as a learning catalyst and 
support. 

Fourth, ..., it can be beneficial to have ad-hoc, shorter, get-together learning 
sessions, which could be held on the business school’s premises themselves, 
offering a useful, efficient facility. ” 



(Lorange 2001 , pp. 2-3) 

Thereby, the key to the success of customisation is the partnership between the business 
school and the corporation. The assumption on the value of customisation is therefore that 
the business school knows of the specific corporate needs and strategic issues. On the 
grounds of this knowledge, the faculty can prepare specified content, and methods, that will 
address and help solve these needs during the customised programme. Given our example 
of strategy process capability, customisation implies that a business school is able to provide 
relevant content that assists in developing the desired strategy skills, knowledge and 
mentality. 

However, there are criticisms with respect to customised programmes on the other hand as 
well. The critics doubt the degree of customisation in executive education and question the 
value for a corporation. 

“More mature management development departments, who are less impressed 
by the high gloss culture of an international management institute, have posed 
the question, on the grounds of their experiences, whether the ‘off-the-peg 
products with adjusted sleeve and trouser lengths’ are worth their money.” 

(Stiefel 2000, p. 21; translated from German) 

We can see that the main criticism refers to the degree of customisation in practice. 
Customised programmes are regarded as an easy and profitable source of revenues for 
business schools. Thereby, critics believe that the label of customisation enables higher 
prices for the same off-the-shelf content. As a result, the customised content would remain 
low with professors not spending enough time and attention to the programmes. Therefore, 
critics discern an inherent struggle between corporations and business schools with respect 
to customisation. Corporations desire their programmes to be as customised as possible. 
This implies that companies require a constant alignment of content in order to respond to 
corporate changes and strategy moves. On the other hand, business schools have other 
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activities as well, for example academic research or MBA courses, and cannot solely focus 
on executive education. Therefore, critics believe, business schools might not always be able 
to deliver highly customised content to corporations (Stiefel 2000). Furthermore, there is a 
claim that the faculty does not have the resources to gather the necessary corporate insights. 

" Business schools are generally at a significant disadvantage in understanding 
the firm’s strategic situation and in creating alignment within the corporation . Most 
business school faculty have not had the opportunity to do an in-depth industry 
analysis and to examine the company’s position within the industry, or to 
understand intimately the firm's strategic intent” 



(Moore 2001, p. 10) 

Overall, the lack of empirical research makes it difficult to form any judgement from an 
academic standpoint. However, given the increasing popularity and relevance of 
customisation in practice, more founded insights into the aspect of customised programmes 
are clearly desirable. Therefore, we consider it as a chance to be able to investigate the 
aspect of customisation as well in the framework of our thesis. 

Summarising the section on customisation - we have addressed the third research question 
(LS3) “What does customisation entail in the area of executive education?”. We have shown 
that customisation embodies a new trend in executive education that comprises of a close 
partnership between business schools and corporations. Customisation addresses the 
content, and partly also the methods, of an executive programme, which are specified to suit 
the needs of the corporation. Little empirical research exists with respect to customisation. In 
practice however, we can discern a growing debate on the advantages and disadvantages of 
customised programmes. 

Thereby, we conclude part 2.2 on the content foundation of our research. We have 
presented the two foci of our thesis, which play a major role in our empirical part later. The 
reader should therefore be familiar with the research matter of our study. Given our 
underlying research intention, the development of a new evaluation approach, the second 
half of the Theoretical Frame now provides an overview of the current state of evaluation 
research. Part 2.3 thus serves as a preparation and as a source of ideas to our conceptional 
contribution in chapter 3. 
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2.3 On Evaluation: The Methodical Approach of the Research 

“To say that there are as many definitions as there are evaluators is not too far 
from accurate . " 



(Franklin & Trasher 1976, p. 20) 

In the following part we look at ways of evaluating the impact of executive education. 
Thereby, our discussion of different evaluation approaches presents the methodical focus 
of our thesis. 

The term evaluation covers a wide range of different research activities (Wottawa & Thierau 
1990). It is understood and applied in different ways by various disciplines (Gotz 1998). In 
the context of our thesis, we define evaluation as the systematic investigation of the value or 
merit of executive training and development for an organisation (cf. Schindler 1979), and 
more specifically, as the assessment of the value of customised executive programmes with 
respect to strategy process capability. 

The objective of part 2.3 is to provide an overview of the different concepts of evaluating 
executive education. Due to the limited scope of our Theoretical Frame, we have decided to 
concentrate on the most common and most influential concepts in evaluation research. 

We approach the field according to a distinction made by Bronner and Schroder (1983). The 
two authors identify two motives for evaluating training and development. 



Pedagogical Motives 


and 


Economic Motives 


• Proof of teaching success 




• Resource generation 


• Motivational incentive for 




• Measurement of resources 


learning 




• Reporting 


• Proof of learning success 




• Proof of efficiency 


• Indicator for educational 






requirements 







Table 2.9: The Two Types of Motives for Evaluation 

Source: Bronner & Schroder 1983, p. 45 (Translated from German) 

We have broadened Bronner and Schroder’s distinction into approaches from the field of 
psychology/pedagogy and the field of finance/controlling (cf. Thierau-Brunner et at. 1999). 
Consequently, in our literature review, we have identified those evaluation approaches of 
both fields, which we found most influential and relevant for our research. As a result, we 
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present 12 contributions in total. Thereby, we feel confident that these 12 evaluation 
approaches can provide a good overview of the state of evaluation research today. 

The Theoretical Frame should assist the reader in putting our new evaluation approach 
(presented in chapter 3) into perspective. With part 2.3 we provide answers to the fourth and 
fifth research questions (LS4 and LS5). " What theoretical and practical approaches exist to 
evaluate training and development?" And “What are specific efforts to assess the impact - 
the return-on^education - of executive education?” 



2.3.1 Underlying Evaluation Concepts in Psychology and Pedagogy 

Evaluation can serve as “Legitimisation", " Improvement ”, “Integration”, “Decision- 
making" and “Forecast". 



(Nork 1991, pp. 39-41) 

In the field of psychology and pedagogy, evaluation plays a central part in understanding the 
dynamics of training and development programmes. It assesses a programme’s educational 
success and provides useful insight on how the intervention can be improved. Hereby, 
evaluation serves several purposes. Nork (1991) lists five different roles of evaluation (see 
above), which emphasise further purposes of evaluation. 

In the following section we introduce a selection of evaluation approaches from the field of 
psychology and pedagogy. Thereby, we distinguish between approaches from the Anglo- 
Saxon literature (subsection 2. 3. 1.1) and the German-speaking literature (subsection 
2. 3. 1.2). The reasons for this distinction are partly historical, partly cultural, partly situational. 

Wottawa & Thierau (1990) point out historical differences in the evaluation research of the 
two regions. In particular the United States started considerably earlier than Germany, 
Switzerland and Austria to investigate evaluation research in the context of education (p. 59). 
Furthermore, evaluation research in the three German-speaking countries experienced a 
weaker and less structured development than the Anglo-Saxon region. The differences in the 
history of evaluation research of the two regions have influenced the scope and content of 
evaluation approaches. 

A second reason for the distinction refers to existing cultural differences. During the 
explorative period of our research, we experienced differences in the evaluation mentality of 
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people in the field. Talking to both American and German practitioners, a different belief in 
evaluability and results significance became apparent. Our experience suggested that 
Americans are more practically oriented, considering evidence as enough to believe in 
evaluation results. German practitioners showed a higher suspicion towards evaluation 
results, requiring watertight proof. 18 To some extent the cultural difference can also be found 
in the literature. 

Thirdly, a situational reason for distinguishing between Anglo-Saxon and German-speaking 
approaches derives from the nature of our research field at DaimlerChrysler (cf. chapter 4). 
Before the merger, Chrysler had primarily been a proud North-American firm and Daimler- 
Benz represented a predominantly German conglomerate. In terms of their human resource 
practices both firms were largely influenced by the culture and the legal system of their 
respective countries. Given our research objective to evaluate executive education at the 
DaimlerChrysler Corporate University, the cultural distinction helped to emphasise the 
cultural heritage of Chrysler as well as Daimler. 

Overall, we acknowledge that the combination of Anglo-Saxon and German-speaking 
contributions only represents a selection of the wider academic work on the field. 

In terms of the evaluation type, we chose to focus on the summative evaluation (Scriven 
1967; Bryson 1984; Noe 1998) due to our research objective of assessing the value of 
executive education. Hence, we include fewer approaches of formative evaluation. 

Summative and formative evaluation are closely related in the field of evaluation research, 
but they differ in terms of their goals as we can see from the following quote. Noe (1998) 
distinguishes between the two types. 

“Summative evaluation refers to evaluation conducted to determine the extent to 
which trainees have changed as results of participating in the training 
programme. ... [It] may include measuring the monetary benefits (also known as 
return-on-investment) the company receives from the programme. Summative 
evaluation usually involves collecting quantitative (numerical) data ... 

“Formative evaluation refers to evaluation conducted to improve the training 
process. ... [It] provides information about how to make the programme better. 



18 Unfortunately, we cannot support these perceived differences with empirical evidence, which thus remain an 
individual impression. 
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Formative evaluation usually involves collecting qualitative data about the 
programme. ” 



(Noe 1998, p. 131) 

There are two underlying reasons for our choice of summative evaluation. On the one hand, 
the research field of summative evaluation has been more dynamic (Haring 2001) and we 
want to contribute to the more dynamic field in evaluation research. On the other hand, the 
question of the summative evaluation is a more interdisciplinary one. The value of executive 
education is of interest to many more stakeholders in the organisation. Coming from a 
general management background, we are thus more interested in summative evaluation. 

In the following two subsections we take a look at a selection of contributions from the Anglo- 
Saxon literature and the German-speaking literature. All contributions come from the field of 
psychology or pedagogy. We portray the major contributions briefly at the end of each 
subsection. Together with the discussion on the concepts from finance and controlling, we 
integrate all evaluation approaches in a comparison in section 2.4.3. 



2.3.1. 1 Approaches in the Anglo-Saxon Literature: The Origins of Evaluation 
Research 

“ The power of Kirkpatrick’s model is its simplicity and its ability to help people 
think about training evaluation criteria. ’’ 



(Alliger & Janak 1989, p. 331) 

As mentioned above, evaluation research has a strong tradition in the Anglo-Saxon 
academic psychology and pedagogy literature. We thus had to make a selective choice due 
to the variety and the large number of approaches. The reason for concentrating on these 
three authors and their approaches is as follows. 

The model of Kirkpatrick is known as one of the most intelligible and most appealing 
evaluation models (Boverie et al. 1994; Bramley 1999; Abernathy 1999; Brettel 1999; 
Thierau-Brunner et al. 2000; Haring 2001). Without doubt it belongs to the most widely 
referenced and discussed models in the Anglo-Saxon literature and has thus contributed 
considerably to the evaluation debate for more than 40 years. Furthermore, it serves as one 
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of the foundations for the development of our evaluation tool in chapter 3. Therefore, it is 
presented and discussed here in some detail. 

Brinkerhoff’s evaluation approach is less common in the literature. However, it is regarded as 
an important contribution to the field of evaluation as well (Eichenberger 1992; Brown 1997; 
Gotz 1998). In particular, Brinkerhoffs circular evaluation process is the reason for 
referencing his work in our thesis. It proclaims a comprehensive evaluation and emphasises 
a continuing evaluation cycle. 

The evaluation ideas of Patton (1978, 1997, 1998) represent the third contribution. Beywl 
and Schobert call Patton the most creative and most demanded evaluator in the United 
States at the moment (2000, p. 80). We discuss Patton due to his utilisation focused 
evaluation approach. This approach introduces the importance of the use of evaluation 
results and procedures - involving a consideration of the different stakeholders of the 
evaluation and their political interests. 



Donald L. Kirkpatrick 



Kirkpatrick’s model is based on a series of articles written in 1959 and 1960. The continuous 
referencing of the model in today’s literature demonstrates the prevailing impact, which 
Kirkpatrick’s ideas still have. The principal idea of the model is that there are four levels at 
which the outcomes of a training intervention can be measured (Kirkpatrick 1959/60, 1975, 
1979, 1996a+b). These are Reaction, Learning, Behaviour and Results. 



4 th Level 
3 rd Level 
2 nd Level 
1 st Level 



Results 



Behaviour 



Learning 



Reactions 



Has the change in behaviour led to measurable 
results for the organisation? 

Did the participants change their behaviour due 
to the training? 



What did the participants learn? 



What did the participants think of the training? 



Figure 2.2: The Four Levels for Evaluating Training 

Source: Kirkpatrick 1959/1960 
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The first evaluation level, Reaction, refers to the immediate response of participants at the 
end of a training programme. Thereby, the first evaluation level often takes the form of a 
questionnaire for participants to fill in (Kirkpatrick 1959/1960), which comprises reactions on 
the quality and usefulness of the training. Participants thus evaluate the form, content and 
method of the training, the trainer performance, and the quality of administration and 
surroundings, i.e. campus, hotel, etc. (cf. Nork 1991, p. 74). 19 

Evaluating reactions is easily done and without great costs (Kirkpatrick 1979; Boverie et al. 
1997). Therefore, it is not surprising that most training practitioners conduct an evaluation at 
the Reaction Level at some form or another. 20 Academics as well as practitioners remain 
uncertain whether the reaction level can provide meaningful results (Eichenberger 1992; 
Nork 1991). Kirkpatrick himself states that the results from this first level are difficult to 
interpret (1979). Nevertheless, the results of Reaction Level questionnaires are commonly 
used to determine the ranking and subsequently the reward for performance of business 
school professors. 21 

The issue of Learning is addressed by the second level. Kirkpatrick defines learning as “the 
principles, facts and techniques that were understood and absorbed by the participants " 
(Kirkpatrick 1979, p. 82). The second level evaluation therefore concentrates on the 
attainment of the programme’s learning objective 22 (Carnevale & Schulz 1990). Different 
evaluation techniques may be used with respect to measuring the effects at the Learning 
Level. The most common form is the paper-and-pencil test (cf. Boverie et al. 1997). In order 
to isolate the learning derived from the intervention, pre-tests and post-tests are proposed 
(cf. Endres & Kleiner 1990). 23 These often include a control group and might take on 
elaborate designs. An extensive overview of control group designs are given by Bronner and 
Schroder (1983) and can be found in appendix 8.8. 

The third level comprises evaluation activities concerning changes in behaviour, which take 
place in the job environment and concern the daily responsibilities of the participants. The 



19 Participants are often asked whether they liked the training and therefore these reaction surveys are often 
nicknamed “happy sheets" (cf. Fischer & Weinberg 1988). 

20 According to estimates, well over 75 per cent of companies evaluate their training at the Reaction Level (cf. 
Foxon 1989; Carnevale & Schulz 1990; Giilpen 1996; Haring 2001). 

21 All three business schools included in the research use Reaction Level participant data to assess the 
performance of their professors, which subsequently influences their performance ratings. 

2 Compare the German term “Lemziel” in educational science and pedagogy. 

23 Public institutions, like schools or universities, regularly conduct assessments at the Learning Level through 
written exams and oral tests. In terms of management education and development Kirkpatrick’s second level 
evaluation of learning is not so often done (Bramley 1999, p. 145). The often abstract and intangible learning 
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Behaviour level therefore addresses the problem of the transfer of learning (cf. Huczynski & 
Lewis 1980; Baldwin & Ford 1988; Foxon 1994; Milheim 1994). In terms of management 
development the Behaviour Level is an important evaluation component. 

“The prime purpose of management development is effective managerial 
behaviour; it is not just knowledge, or attitudes, although these clearly can be 
significant contributors to effective behaviour " 



(Mumford 1995, p. 5) 

The transfer of learning plays an important role because training and development realises 
most of its impact through real life application of learning. Kirkpatrick’s third level focuses on 
the transfer and on the changes of behaviour that matter in terms of job performance. 

The fourth level in the evaluation process according to Kirkpatrick comprises the results or 
the impact on the organisation (1979). Kirkpatrick implies that the results on this level are 
expressed in quantitative terms. The attained improvements are stated in organisational 
gains 24 and are weighted against the cost of the training. For many academics and more so 
for practitioners the data from the Results Level are the most interesting as they allow the 
impact of training and development to be formulated in the financial language of business. 
Data from the Results Level could answer the question of the value of training and 
development and would enable a fair discussion on the worth of the investment in education 
(Phillips 1996a+c, 1997a+b, 1999). 

Considering the four levels, the higher the level of the model, the more difficult it becomes to 
evaluate. Kirkpatrick himself states that “evaluation of training programmes in terms of the 
job behaviour is more difficult than the reaction and learning evaluations" (1979, p. 86). The 
causalities and the relevant influencing factors are difficult to understand when looking at the 
Behaviour or Results Levels (Nork 1990; Tziner et al. 1991; Eichenberger 1992; Bramley 
1996). The positive as well as the negative changes in behaviour and results can be a result 
of the programme, but they may be caused by other environmental factors as well. 

We consider it necessary to take a look at the further development of the model. In his 
original work, Kirkpatrick did not speak of any causality or correlation between the four levels. 



objectives of management development, as for example open-mindedness, make it difficult to identify the degree 
of learning. 

4 Examples of results criteria include managerial effectiveness, productivity gains, customer satisfaction, 
employee morale, cost savings and profitability. 
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However, soon after its publication academics have extended Kirkpatrick’s model and 
implicitly assumed three dynamic relations. 

Firstly, Kirkpatrick’s model is interpreted that the order of the four levels reflects a hierarchy 
of the value of information provided. Noe and Schmitt use the term hierarchical model (1986, 
p. 497). The hierarchy implies that the four levels are ranked - the Reaction Level being the 
lowest with the least information value and the Results Level being the highest with the most 
valuable information content. 

Secondly, Hamblin (1974) assumes a causality between the levels. In his view, outcomes on 
the Reaction Level influence outcomes on the Learning Level, which then determine 
outcomes on the Behaviour Level and so forth. Kirkpatrick does not assume the same 
causality, “it is important to recognise that favourable reaction to a programme does not 
assure learning’’ (1979, p. 82). Hamblin (1974) therefore develops his own model - clearly 
derived from the model of Kirkpatrick - which contains five levels: Reactions, Learning, Job 
Behaviour, Organisation and Ultimate Value. Hamblin assumes a clear causality between the 
five levels in his model. As a result of Hamblin’s work, causality between the different levels 
has also often been assumed for Kirkpatrick’s model as well. 

Thirdly, academics assumed that the levels are positively correlated (cf. Newstrom 1978) so 
that positive reactions would lead to positive learning, which would then result in positive 
changes in behaviour. In particular, these common assumptions generate critical reflections 
on Kirkpatrick’s model. 

Alliger and Janak’s (1989) and Alliger et a/.’s (1997) meta-analysis of the extended model 
has received considerable attention (Eichenberger 1992; Giilpen 1996; Noe 1998). The 
meta-analysis comprises a literature review of over 200 articles, which all apply Kirkpatrick’s 
model. Alliger and Janak focus on reported correlations between Kirkpatrick’s levels in order 
to judge the validity of the model. To be exact, they refer to the extended Kirkpatrick model of 
Hamblin (1974 - cf. Newstrom 1978; Noe & Schmitt 1986). In their research Alliger and 
Janak test the following assumptions of the extended model. 

“(1) The levels are arranged in ascending order of information provided. (2) The 
levels are causally linked. (3) The levels are positively inter-correlated. ” 

(Alliger & Janak 1989, p. 331) 

Alliger and Janak’s results (in 1989 as well as in the study of Alliger et al. in 1997) provide 
little evidence for the logic of the extended Kirkpatrick model of Hamblin. Both studies come 
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to the conclusions that none of the three assumptions can be taken for valid. In their opinion 
there is no hierarchy in the evaluation levels. "... it is not dear that all training in organisations 
is meant to effed change at all four levels' 1 (Alliger & Janak 1989, p. 332). Alliger and Janak 
(1989) discover a different correlation structure, which rejects Hamblin’s assumptions. 

The most important findings are reinforced by the later study as well (1997). There is no 
proven positive correlation between Reaction and Learning (cf. figure 2.4). Secondly, levels 
two to four are related in more complex and reciprocal ways. Alliger and Janak’s results 
provide important new insights on the structure of the original Kirkpatrick model. At the same 
time they call for more careful consideration of possible linear causal links by disproving the 
extended assumptions of later authors. 

We have added a suggestion for an extended model, which amends the findings of Alliger & 
Janak (1989). Our model only serves the purpose to suggest a new perspective, but cannot 
provide any empirical foundation. We propose that Reaction could influence Learning in a 
positive and a negative way. Often, it is a very bad or traumatic experience that causes a lot 
of learning. This would imply that there is an inverse relationship. Secondly, we wonder 
whether a lot of Learning might not influence Reactions inversely. It seems likely, that if a 
person were able to learn a lot from a programme, he would report very positively about it. 




Figure 2.3: Different Causality Structures for the Kirkpatrick Model 

Source: Derived from Alliger & Janak 1989; Alliger et at 1997 
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In conclusion - we consider the Kirkpatrick model as a simple and universal framework to 
think about evaluation. The model provides a clear structure to the process of evaluating 
training and development and helps to classify evaluation results. However, we have noticed 
that the model tends to mislead evaluators in terms of the simplicity of interrelationships with 
respect to its evaluation levels. This is reinforced by the tendency to focus on the Results 
Level (Abernathy 1999) and thereby misapply evaluation resources at the wrong end. We 
conclude that, if taken as it was originally conceived by Kirkpatrick, the model still adds value 
to the evaluation research today by helping understand the multi-level dynamics of training 
and development. 

Robert O. Brinkerhoff 

The work of Brinkerhoff (1987) is chosen to be included in the framework of our thesis 
because of his comprehensive evaluation model. Although the model receives less 
attention in the Anglo-Saxon literature, Brinkerhoff represents an elaborate development of 
Kirkpatrick’s ideas. The strong results-oriented aspects of evaluation by Kirkpatrick are 
combined with a strong formative, improvement-oriented part. 




Figure 2.4: The Six Stage Model as a Cycle 

Source: Brinkerhoff 1987, p. 27 



54 




Brinkerhoffs model investigates in the pay-off (cf. p. 14) as well as in the opportunities for 
improving the training and development intervention. The model thus includes a summative 
and a formative evaluation approach. Brinkerhoff perceives the assessment as an iterative 
process and adds thereby the idea of a circular evaluation approach to the research 
discussion. 

Brinkerhoff emphasises that the evaluation stages are not simply accomplished once and 
then abandoned for the next stage, never to be returned to (p. 31). The stages are interlinked 
and depend on each other. Brinkerhoff suggests an evaluation cycle, which is repeated over 
and over again - continuously improving the programme. Furthermore, the six stages can be 
classified into three temporal phases - before, during and after the programme - in contrast 
to Kirkpatrick, whose model focuses primarily on the time after the programme. 

Stage I and II represent the formative part of the evaluation model. The first stage comprises 
the needs analysis for a particular HRD programme in order to answer the question Does a 
need for training exist? Is there a problem or opportunity where training can make a 
worthwhile difference? (cf. Brinkerhoff 1987). Its purpose is to identify its intentions and 
objectives (p. 39). Stage I generates the information as to which specific skills, knowledge or 
mentality should be changed to achieve an impact and whether these are achievable through 
training and development. In other words, the role of stage I is to look ahead in order to plan, 
critically assess and decide on the content and the methodology of training and 
development. Brinkerhoff suggests the following procedures - organisational audits, 
performance analyses, record analysis, observation, surveys, study of research, document 
reviews and context studies. 

Stage II continues the formative evaluation part (before a programme) and aims at 
translating the insights of the first stage into a concrete programme design. Primarily it 
assesses a given design’s practicality, theoretical soundness and responsiveness of possible 
alternatives (p. 27). The purpose is to give answers on the question Is the training design 
good enough to implement? Brinkerhoff hence emphasises the importance of the process of 
designing the intervention. He mentions literature review, expert reviews, panels, checklists, 
site visits, pilot tests and participant reviews to support stage II. 

The third stage places its focus on the process during the programme. It serves primarily as 
a monitoring role. The comprehensive nature of the approach becomes apparent. Brinkerhoff 
includes a practical feedback function in the evaluation model in order to respond to How is 
the training going? Is it working? In terms of methods he suggests observation, checklist, 
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trainer and trainee feedback and record analysis. The third part closes the formative part of 
the evaluation model and at the same time opens the summative part. Stage III does not only 
consist of monitoring the programme’s realisation, but also of gathering feedback about the 
reactions of participants after the programme (p. 30). Hereby, Brinkerhoff refers directly to 
Kirkpatrick’s ideas. Stages III (partly), IV, V and VI are very similar to Reaction, Learning, 
Behaviour and Results. 

Stage IV tests the learning of participants with respect to improvements in skills, knowledge 
and attitudes - Did the participants learn what they were supposed to do? Brinkerhoff 
mentions knowledge and performance tests, observation, simulations, self-reports and work- 
sample (product) analyses as possible evaluation methods. Consequently, stage IV findings 
lead on to the analysis of stage V. 

The fifth stage assesses how much and how well acquired skills, knowledge and attitudes 
are translated into intended on-job behaviour. Brinkerhoff investigates the evaluation of the 
durability of these changes. Positive findings at the fifth stage indicate that organisational 
effects can be expected. Brinkerhoff suggests the following methods - self-, peer-, and 
supervisor reports, case studies, surveys, site visits, observation and work sample analyses. 

The sixth stage represents the core of the summative evaluation part of the model. The value 
of the organisational effects are the focus of the last stage. The final stage assumes that the 
training and development has worked at all stages so far (i.e. achieved positive results). 
“People learned something and are in fact using what they learned” (p. 30). Brinkerhoff 
includes organisational audits, performance analyses, record analyses, observation, surveys, 
document reviews, panel reviews and hearings, and cost-benefit comparison. The role of 
stage VI is to convert the changes of behaviour into more tangible, if not monetary values. 
Ultimately the worth of the training and development is determined by comparing the benefits 
to the costs (p. 30). 

Taken as a whole, Brinkerhoffs model proves beneficial to the understanding of the field of 
evaluation. First of all, the different stages of the model reflect an ideal design and 
implementation process for a programme. Brinkerhoff perceives evaluation as being a vital 
part of this process, and not as something that is done in isolation afterwards. His message 
is clear - a successful evaluation process starts early in the design stage. Secondly, 
Brinkerhoff contributes the idea of continuous improvement to evaluation, i.e. he perceives 
the process as circular and iterative. He observes that the real world is fraught with error (p. 
xiv) and that it is therefore unlikely that the first evaluation attempt will be perfect. 
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Eichenberger (1991) criticises and amends Brinkerhoffs ideas by the insights from the Alliger 
and Janak study (1989). Given their asymmetric correlation structure of Kirkpatrick’s four 
levels, Eichenberger corrects (in his view) the summative part of the model. He questions the 
link between stage III and IV. Furthermore, Eichenberger (1991) perceives additional links 
between stage II and IV (design causes learning), between IV and VI (learning causes 
payoff), and between VI and V (reverse link - payoff initiates usage). We remain doubtful 
about these corrections. Eichenberger assumes clear correlations between the stages (his 
own interpretation of the arrows in the Brinkerhoff model). We understand the graphical 
illustration of the model rather in a process way - stage I proceeds to stage II (instead of 
Eichenberger’s stage I causes stage II). Moreover, Eichenberger does not produce any 
evidence for or against his corrections. Therefore, his interpretation of Brinkerhoffs work 
should be conditioned by the original text of the model (1987), which indicates that the 
stages are meant as a series of consecutive process steps (cf. Brinkerhoff 1987, p. 16). 

Overall, we consider Brinkerhoffs work appealing due to its detailed process description. His 
model takes a stand against real life problems and aims to provide a useful tool to 
evaluators. His work serves as a compendium for evaluation methods and has therefore 
influenced the selection of methods of our new evaluation approach. However, Brinkerhoff 
himself admits that his model 7s an ideal”, which might be difficult to implement in a real 
world of tight deadlines and limited resources (p. 233). Hence, Brinkerhoff perceives his work 
as “a direction and a goal for excellence”. 

Michael Q. Patton 

Patton’s work does not comprise an evaluation model in the sense of Kirkpatrick’s four 
levels, but contributes several significant ideas to the field. He combines both academic 
evaluation research and extensive field experience. Patton’s practical insights are the reason 
for including him in the Theoretical Frame. His work contributes to the conceptualisation and 
the development of the new evaluation tool presented in chapter 3. From the variety of his 
work, we highlight firstly the premise of the utilisation-focused evaluation, secondly the 
respect for different evaluation stakeholders, and thirdly the need for a comprehensive 
approach. 

A first important contribution of Patton is his premise of the utilisation-focused evaluation 
approach. Its axiom is clearly stated as “evaluations should be judged by their utility and their 
actual use” (Patton 1997, p. 20). Patton thereby understands the evaluation process itself as 



57 




an investment, which must turn out useful and worthwhile for the stakeholders of the 
evaluation. He takes a down-to-earth perspective and defines use as follows. 

"Use concerns how the real people in the real world apply evaluation findings and 
experience the evaluation process . Therefore, the focus in utilisation-focus 
evaluation is on intended use by intended users.” 



(Patton in Hinn 1998, p. 1) 

Patton criticises the many attempts that are made to devise a perfect evaluation instrument 
in theory. 25 The focus rests on actual use, and thus not only on usability. To ensure actual 
use, an evaluation process must identify and formulate the goals of the intervention 
specifically (Joas & Patton 1999). According to Patton, goals are defined anew by each 
training and development situation and depend on the goals and interests of the different 
stakeholders (Patton 1997). 

Secondly, Patton emphasises the existence of different stakeholders in an evaluation 
process. In Patton’s view, evaluation is always a political process (Patton 1997). “In any 
evaluation there are many potential stakeholders and an array of possible uses” (Patton in 
Hinn 1998, p. 1). The notion of stakeholders can contain different interest groups. 

On the one hand, there are the different users of the evaluation - the evaluators, the 
trainers, the organisers, but also the sponsors of the intervention. They are concerned with 
respect to the impact of the evaluation. But on the other hand, stakeholders can also 
comprise the customers of the training and development - the participants and their 
superiors. Their stake in the evaluation process is that their specific needs are respected (cf. 
Brinkerhoff 1987). Participants regard the evaluation process as an opportunity to 
communicate their needs and to improve training and development. Patton explains that an 
evaluation cannot serve all and involves therefore careful manoeuvring, especially in real life 
environments. 26 

“Since no evaluation can be value-free, utilisation-focused evaluation answers 
the question of whose values will frame the evaluation by working with clearly 



25 An example for a highly complex model is Hoffmann’s INDEL-Evaluation Model (1977) as shown in Bronner & 
Schroder 1983, p. 63. 

6 Often evaluation is regarded as a threat by training professionals. The assessment process controls their work 
and can have negative consequences for them, when failure or mistakes are discovered. High personal stakes 
indicate the likelihood of a complex political process (Clarke 1999). 
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identified, primary intended users who have the responsibility to apply evaluation 
findings and implement recommendations. " 



(Patton in Hinn 1998, p. 1) 

The practical consequences of Patton’s insight are far reaching in the evaluation process. 
The priority given to certain stakeholders’ interest determines the focus of the evaluation. 
The specific focus influences the results of the evaluation according to the maxim “what gets 
measured gets done ” (Patton 1998, p. 230). For example, the head of controlling is defined 
as the most important stakeholder. Patton’s logic states, that is it quite likely that the specific 
interest of the controller influences the objectives of the evaluation. Consequently, 
predominantly financial measures are used in the assessment process. These measures can 
have important repercussions for the subsequent design of training and development. 27 
Patton thereby emphasises the responsibility of the evaluator (Joas & Patton 1999). Badly 
defined goals of an evaluation might cause dysfunctional behaviour and might lead to the 
opposite of what is intended. 

The third of Patton’s contributions considered here is the importance of a comprehensive 
approach. Patton advises to avoid a piecemeal approach to evaluation and speaks against 
isolated evaluation techniques. 

“Real world circumstances are too complex and unique to be routinely 
approached through the application of isolated pearls of evaluation wisdom. What 
is needed is a comprehensive framework within which to develop and implement 
an evaluation with attention to use built in. ’’ 



(Patton 1997, p. 20) 

Patton pursues a contingency approach to evaluation. Evaluation should be 
comprehensive with respect to the different outcomes of the training and development. Each 
training and development scenario demands a selection of appropriate evaluation content 
and methods anew. 

Furthermore, according to Patton, most evaluators have come across the obstacles of 
corporate realities, which affect the implementation of a sound experimental design. 



27 If the success of a training is merely defined by the achieved profit for the training department, it seems likely 
that soon the design of programmes will prefer popular and profitable elements. Training would become more 
profit oriented. 
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Controlling for influencing variables becomes almost impossible in the context of very 
heterogeneous participant groups in complex environments (cf. Bronner & Schroder 1983). 
Patton illustrates that trade-offs are inevitable. Evaluators must strike a balance between an 
elaborated evaluation design and its feasibility in terms of time and resources. 

In conclusion - we agree with Patton that the field of evaluation research should be reminded 
of its real life connection. The world is ‘messy’ and therefore no one isolated perspective 
provides the solution to evaluation. Faced with real life complexity, it is thus important to 
remember the utilisation of the evaluation results, the political environment and the need for 
a broad perspective rather than a limited point of view. 

Concluding subsection 2.3.1. 1 - we summarise the three approaches presented in the 
following table. 



Evaluation 

Characteristics 


Kirkpatrick (1959/60) 


Brinkerhoff (1987) 


Patton (1997) 


Evaluation Focus 


Level-oriented 
Results of education 


Process-oriented 
Results of education 


Action-oriented 

Utility and actual use of 
evaluations 


Type of Evaluation 


Summative 


Summative & formative 


Not specified 


Level of Analysis 


Individual & organisational 


Individual & organisational 


Variable 


Evaluator 


Training directors 


Training professionals, 
evaluators 


Users (evaluators, trainers, 
organisers, sponsors) 


Stakeholders 


Participants 


Evaluation audience 
(instructors, training design 
staff, participants, training 
manager) 


Customers (participants, 
superiors) 


Evaluation Process 


Evaluation and 
measurement at four levels 


Continuous, iterative 
evaluation cycle of six 
stages 


Comprehensive framework 
designed for each 
evaluation situation anew 


Evaluation Cycle/ 
Timeframe 


Before and after 


Before, during and after 


Variable 


Evaluation Methods 


Qualitative feedback 

Quantitative/statistical 

analysis 


Qualitative audits, 
observations, surveys, 
interviews and reviews 

Quantitative checklists, 
tests and comparisons 


Not specified 


Strengths 


Structure 

Simplicity 

Comprehensiveness 


Emphasis on continuous 
evaluation 

Wealth of methods 
Process orientation 


Recognition that no 
evaluation is value-free 

Consideration of the 
political aspects of 
evaluation 

Contingency approach to 
evaluation 



(continued on the next page) 
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Evaluation 

Characteristics 


Kirkpatrick (1959/60) 


Brinkerhoff (1987) 


Patton (1997) 


Weaknesses 


Ambiguity of the 
relationship between levels 


Ideal model for a perfect 
evaluation world - missing 
link to reality of evaluations 


No concrete evaluation 
methods/tools provided 



T able 2.10: Summary of the Anglo-Saxon Approaches in Psychology/Pedagogy 

We close subsection 2.3. 1.1 by reviewing what we have learned from the three Anglo-Saxon 
approaches from psychology/pedagogy, i.e. the ‘So what?’. 



Our discussion provides some first answers to the fourth research question (LS4). Evaluation 
can be done at different measurement levels (Kirkpatrick 1979; Brinkerhoff 1987). Thereby, 
we discover that evaluation presents a multi-dimensional problem. The value of training 
and development is not only created through one channel (linear cause-and-effect), but 
evolves through a complex interaction of factors (multiple and interlinked cause-and- 
effects). Therefore, it is important that an evaluation takes a comprehensive approach by 
including the most important dimensions of training and development. Both Kirkpatrick and 
Brinkerhoff show the necessity for multi-dimensionality in their approaches. 

Furthermore, the discussion of the three approaches demonstrates the importance of 
transfer. In particular, Kirkpatrick includes the assessment of transfer in his model. Value 
can only be created if the learning is used and applied. The return for the organisation can 
only come through the participant(s) and the application of their learning. 

We find Kirkpatrick and Brinkerhoff helpful in understanding the nature of evaluation. Both 
provide very structured approaches that outline the different conceptional levels, or steps, 
that one can take to evaluate training and development. In contrast, Patton points out the 
practical implications of evaluation. We learn about the difficulty to evaluate due to possible 
conflicts of interests in real life. Patton advises us to anticipate the ‘messiness’ of real 
evaluations. He reminds us that evaluation should not be done for the evaluation’s sake, but 
should be of real value to its intended users. 

Therefore, in section 2. 3.1.1 we have learned how evaluation can be done. The age of 
Kirkpatrick’s approach also demonstrates that no-one-best way has been found yet and that 
the evaluation research is still looking for new approaches. Furthermore, we have touched on 
the difficulties that can arise in real life evaluations. Next, we take a look at German-speaking 
contributions to the field of evaluation. 
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2.3.1. 2 Approaches in the German-Speaking Literature: The Emphasis on 
Systems and Transfer Analysis 

“At the beginning It should be noted, that there is no magic evaluation technique, 
which solves all the problems. ” 



(Stiefel 1 974, p. 8; translated from German) 

Like the Anglo-Saxon literature, the German-speaking evaluation research embraces a large 
number of studies and contributions. In the following subsection we introduce five German- 
speaking writers (or groups of writers) who we consider the most relevant in the context of 
our thesis. The chosen German-speaking approaches differ in their content and scope from 
the Anglo-Saxon contributions. In particular, the German-speaking writers evaluate corporate 
education more from a systems perspective and place a greater emphasis on the evaluation 
of transfer of learning. 

As one of the first in the German-speaking literature, Stiefel (1974) 28 derived the need for 
evaluation. The main contribution of Stiefel consists of various definitions and evaluation 
concepts, which have shaped the later evaluation research in the German-speaking 
literature. Stiefel’s work has remained a standard reference for many evaluation studies (cf. 
Beywl and Schobert 2000). 

A second classic among the German psychology and pedagogy approaches is the work of 
Bronner and Schroder (1983). They have produced an evaluation compendium of how to 
assess the success of corporate education. Bronner and Schroder’s work was chosen 
because they regard evaluation as part of a wider system of Human Resource Development. 

Beywl (1988) is another author, who has shaped the German evaluation landscape. Beywl 
advances the common understanding of evaluation by reflecting on the underlying theoretical 
basis of evaluation models. 

The fourth pair of authors consists of Nork (1991) and Gotz (1998), who both emphasise the 
importance of the transfer of training and development. They are included in the subsection 



28 Professor Rolf Th. Stiefel is Swiss, but has published most of his work in German. Therefore, we classify him 
as part of the German-speaking literature. 
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because their work reflects a growing evaluation effort in the training and development 
industry to achieve transparency of the transfer of learning. 

Finally, we include a study of Haring (2001), who conducted a meta-study of the evaluation 
practices in large German companies. We chose Haring as the last author because she 
contributes a ‘snapshot’ of the current state of the art of evaluation practice in Germany. 

Rolf Th. Stiefel 

Stiefel’s work (1974) can be regarded as a classic, which serves to this day as a basis for the 
German-speaking evaluation research literature. At the time when it was published, it 
provided the necessary structure to the growing field of evaluation. Thereby, Stiefel explains 
the evaluation process from three perspectives. 

Firstly, Personal Types of Evaluation perceive evaluation as performed by different people. 
Stiefel gives answer to the question, who evaluates what (or whom). Participants, teachers or 
training directors evaluate different aspects of training and development because they have 
different information needs. Secondly, Temporal Types of Evaluation consider the process of 
the evaluation. Stiefel deals with the timing of an evaluation and explains the rationales for 
gathering data at different points in the evaluation process. Thirdly, Stiefel distinguishes 
Partial Types of Evaluation , which indicate the type of evaluation variables examined. 
Thereby, educational variables, like the teaching material, are distinguished from 
organisational variables, like the organisational climate. 

As one of the first (including the Anglo-Saxon region), Stiefel placed evaluation at the heart 
of a process that derives management training and development from the corporate mission 
and goals. Thereby, he anticipated the underlying logic of SHRD (as seen in subsection 
2 . 2 . 2 . 1 ). 

According to Stiefel’s model (cf. figure 2.5 on the next page), the evaluation should include 
not only the realisation of training and development. The process should also include a 
review, in how far the HRD goals and programmes are aligned to the corporate goals and 
needs. Thereby, Stiefel emphasises a wider systems perspective of evaluation and places 
a new responsibility onto the training and development staff to link content and methods to 
corporate goals. In the terminology of today, Stiefel devised the first strategic evaluation 
model. 
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Figure 2.5: The Process of Evaluation 

Source: Stiefel 1974, p. 12 (Translated from German) 

In terms of possible criticism of Stiefel’s model, we pose the question, whether the influence 
of strategy to the evaluation of training and development is only of a one-way nature. We 
consider it possible, that the realisation and the evaluation of training can affect the 
organisation needs, and can even influence corporate goals. For example, the newly gained 
skills of employees can be a new competitive advantage in the strategic scenario, which 
would consequently change the organisational needs. 

Another reason for including Stiefel in the Theoretical Frame is his recognition of the limited 
application of experimental designs. Stiefel recognises the need for quasi-experimental 
design (cf. Campbell & Stanley 1966), but still voices concern on its feasibility. At the same 
time Stiefel claims that it is probably impossible to calculate a well founded return-on- 
investment for management training and development (cf. Stiefel 1974, p. 8). In his 
perspective, evaluation should aim at bringing light to the black box of executive training and 
make its structure more understandable instead of measuring its monetary output. 
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Concluding - our interest in Stiefel’s work is founded on the links that he draws between the 
corporate goals and the need for evaluation. His model thereby demonstrates a wider 
strategic and organisational purpose of evaluation. 

Rolf Bronner and Wolfgang Schroder 

In the context of our thesis, the most interesting piece from the work of Bronner and 
Schroder (1983) is their illustration of the complexity in achieving learning success. 

Figure 2.7 is included in this subsection, since it provides a good overview of the complexity 
of the teaching and learning environment of executive education programmes. Bronner and 
Schroder identify four key variables, which influence successful learning (programme, 
participants, trainers and situation). All four are interlinked and influence the learning 
success. 



Influence 




Influence 



Figure 2.6: The Determinants of Learning Success 

Source: Bronner & Schroder 1983, p. 181 (Translated from German) 



Although simplified, the overview reminds us of the many variables of training and 
development. Bronner and Schroder emphasise the importance of a comprehensive systems 
view. There is not only one isolated variable that determines the learning success, but it 
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results from an interplay of four variable clusters. Subsequently, the four variables influence 
the scope of the evaluation process and should therefore be considered independently. 

We consider the work of Bronner and Schroder as important since they portray the 
complexity of training and development and thereby demonstrate the difficulty to identify the 
underlying factors for success or failure. Their illustration shows that an evaluation has to 
deal with a complex and multi-dimensional network of causes and outcomes. The only 
criticism that we have concerning their model is that it is dosed off. It seems that no external 
variables, as for example organisational or financial variables, influence the learning 
success. We consider it likely that additional complexity is introduced due to the 
embeddedness of education in the corporate framework (Stiefel 1974). 

Wolfgang Beywl 

The work of Beywl provides a classification grid, which assists us in understanding the 
theoretical basis of different evaluation models (Beywl 1988; Beywl & Schobert 2000). Beywl 
opens a way to an analysis of the evaluation process from an epistemological perspective. 

Following Beywl’s classification, the first characteristic of an evaluation model is its structure. 
It can be emergent or pre-ordinate (cf. Beywl 1988). The structure therefore indicates, 
whether the evaluation constantly refines its focus in order to explore a subject matter 
(emergent). Or whether the questions are well defined before the evaluation and are kept the 
same (pre-ordinate). 

Secondly, Beywl considers the alternatives of an objective versus a subjective evaluation. He 
distinguishes between a value-free evaluation and a value-laden evaluation. However, 
Beywl reminds the reader that most evaluations are value-laden and thus cannot be 
considered as objective. 

The third classification concerns the underlying methodology of the evaluation. According to 
Beywl, an evaluation can be done in a deductive way, by defining hypotheses and applying 
statistical analysis in an experimental way in advance. Or - on the other hand - the 
evaluation can be done more inductively, by following more qualitative, hermeneutic 
methods. 

Finally, Beywl considers the degree of democracy in an evaluation approach. The degree of 
democracy describes how far the stakeholders of the training and development are involved 
in the evaluation through primary data collection methods (high degree of democracy) or to 
what extent secondary data is used (low degree of democracy). 
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Concluding - our brief presentation of Beywl’s classification typology should assist the reader 
in understanding the later presented evaluation approach. 



Manuela E. Nork and Klaus Gotz 

In terms of the general field of evaluating management training and development, the most 
recent literature focuses on the importance of transfer (Nork 1991; Riedel 1992; Ulrich 1995; 
Gotz 1998). Nork (1991) classifies the transfer process into two stages. The first one 
describes the transfer of learning from the training to the workplace (Transfer to the System) 
and the second one illustrates the possible transfer within the environment of the workplace 
(Transfer within the System). The first stage of transfer therefore addresses the issue of 
individual learning, whereas the second stage includes notions of organisational learning 
(Argyris & Schon 1978, 1996; Schein 1993). Given a systems perspective, the transfer of 
learning is not limited to one individual, but can be multiplied through the individual as well. 

Nork (1991) considers the transfer as the key of successful executive education. 
Management training and development often addresses changes in attitudes and behaviour. 
Therefore, Nork emphasises the importance of attempting to change attitudes in order to 
guarantee a sustained change in behaviour. She clearly distinguishes between short-term 
and long-term changes in behaviour, whereas the long-term change is much more 
dependent on situational and environmental factors of the workplace. Nork stresses that only 
long-term changes can ensure that the investment of training and development pays off (cf. 
also Riedel 1992). 

Gotz contributes to the discussion on the transfer by proposing a new model. His Model of 
the Didactic Field concentrates on the context of transfer. Gotz distinguishes between 
primary and secondary elements in the didactic field and stresses their importance for 
ensuring a positive learning transfer. Primary elements include content of learning, method, 
influence of trainers, learning climate, individual interests in training and development. 
Secondary elements include area of tension, the relationship between the individual and the 
system, interaction and interdependence (Got z 1998). Gotz’s model focuses on the transfer 
as the most important criterion of success in contrast to earlier studies, which emphasise the 
success of learning (cf. Bronner & Schroder 1983). Furthermore, Gotz takes a dynamic look 
at the didactic system and illustrates the change in variables over time and space (1998, 
p.163). 

Nork and Gotz’s work is interesting for our thesis as it illustrates the manifold variables that 
come into play in the transfer from an executive programme to the work place. We liked their 
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understanding of evaluating the exchange of learning across systems, i.e. from the ‘system 
of the programme’ to the many ‘workplace systems’. However, we consider the aspect of 
transfer as only a part in a wider set of channels through which value is created. For 
example, the effect of networking is difficult to describe by the notion of ' transfer \ 

Karin Haring 

Given the fairly theoretical considerations of some of the authors above, Haring (2001) 
provides us with a practical perspective on evaluation. Haring conducted a study among 20 
large-scale German companies with respect to their evaluation practices. The objective of 
her study was “a survey and description of the toois used for the evaluation of corporate 
educational programmes for executives" (p. 73). 
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Figure 2.7: The Process of Education and the Different Evaluation Phases 

Source: Haring 2001 , p. 73 (Translated from German) 



As the theoretical foundation of her study, Haring integrates the model of Kirkpatrick into the 
wider process of education. Her use of Kirkpatrick demonstrates the continuing relevance of 
his model. Furthermore, Haring’s model relies on the structure of Brinkerhoffs model as well. 



With her model, Haring provides the most comprehensive evaluation approach. Her 
contribution can be seen as combining different evaluation models into one. However, we do 
not agree fully with her distinction of the different types of evaluation. Haring attempts to find 
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a specific evaluation term for each step in her framework. Thereby, the distinction of the 
different terms seems problematic to us. For example, we regard formative and summative 
as broad typologies, which include a process evaluation or a cost-benefit analysis 
respectively. Summative evaluation is not only confined to the assessment of reactions as 
Haring puts it. 

Haring’s study reveals that 100 percent of the companies surveyed evaluate on the Reaction 
level, whereas only 25 percent conduct an assessment at the Results level. 29 However, 90 
percent of the companies surveyed considered the evaluation at the Results level as 
important or very important (Haring 2001). Her findings deliver an interesting piece of 
information with respect to evaluation practice. We can see that a lot of evaluation is done, 
but very few conduct a systematic approach that includes all four levels of the Kirkpatrick 
typology (Hackett 1997). The most interesting contribution of Haring’s study is her overview 
of the various evaluation tools used in practice. Grouped according to Kirkpatrick’s four 
levels, Haring found the following instruments in use (table 2.11). 



Level 1: Reactions 


Level 2: Learning 


Level 3: Behaviour 


Level 4: Results 


• End-of-programme 
questionnaire 

• Closing group 
discussion 

• Interviews with 
participants 

• Interviews with trainers/ 
professors 

• Interviews with 
superiors 

• Observation of 
participants during the 
intervention 

• Programme report 


• Questionnaire survey of 
participants 

• Interviews with 
participants 

• Self-observation/ 
external observation 

• Interviews with trainers/ 
professors 

• Interviews with 
superiors 

• Role-play during the 
programme 

• Programme report 

• Project work 

• Planning game 


• Questionnaire survey of 
participants 

• Interviews with 
participants 

• Action plan 

• Interviews with 
superiors 

• Interviews with 
subordinates of 
participants 

• Follow-up event 

• Observation of 
participants at their 
workplace 


• Questionnaire survey of 
participants 

• Interviews with 
participants 

• Interviews with 
superiors 

• Interviews with 
subordinates 

• Interviews with 
customers of 
participants 

• 360° feedback 

• Project work 

• Use of indicators like 
length of stay, turn- 
over, absenteeism 

• Forecasts/estimates by 
participants and their 
superiors 



Table 2.1 1 : Evaluation Instruments Used in Practice 

Source: Haring 2001 , p. 74 (Translated from German) 



29 75 percent of the companies surveyed evaluated the training and development at the Learning level, and 65 
percent conducted an assessment at the Behaviour level. 
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Haring emphasises that the measurement at the Results level involves predominately 
interviews or questionnaires at the respective companies. The monetary assessment of the 
impact of the programme thus remains subjective , relying on the perceptions and estimates 
of participants and superiors. Haring concludes: 

“It is true that the return-on-investment (ROI) of educational interventions, which 
is frequently described in the literature, is widely demanded in practice, but it is 
regarded as not feasible . " 



(Haring 2001, p. 76; translated from German) 

Finally, Haring adds another interesting observation. None of the companies surveyed have 
assigned a specific budget for their evaluation activities. In most cases, evaluation is funded 
by the overall budget on the educational interventions. Haring’s finding is line with an earlier 
work, that criticised the absence of a separate budget. Eichenberger (1990) wrote “millions 
for corporate education, pennies for evaluation ” (title of his article). The lack of evaluation 
might often be explained by the shortage of funds. 



Concluding subsection 2.3.1. 2 - we can summarise the different approaches in the following 
table. 



Evaluation 

Characteristics 


Stiefel (1974) 


Bronner & 
Schroder (1983) 


Beywl (1988) 


Nork (1991) & 
Gotz (1998) 


Haring (2001) 


Evaluation Focus 


Process-oriented 

Success and 
transfer of learning 


Process-oriented 
Learning success 


Theory-oriented 

Theoretical basis of 
evaluation models 


Transfer-oriented 


Process-oriented 

Results, transfer, 
cost-benefit, 
organisational audit 


Type of Evaluation 


Summative 


Summative & 
formative 


Value-free vs. 
value laden 


Summative 


Summative & 
formative 


Level of Analysis 


Educational & 
organisational 
variables 


Programme 

Situation 

Trainer 

Participants 


Primary data, 

involving 

stakeholders 

Secondary data 


Individual learning 

Organisational 

learning 


Kirkpatrick’s four 
levels 

Brinkerhoffs 
circular evaluation 
model 


Evaluator 


Personal types of 
evaluation 

• Participants 

• Teachers 

• Training 
directors 


Not specified 


Not specified 


Not specified 


Not specified 


Stakeholders 


(see above) 


Trainer, 

participants 


Not specified 


Not specified 


Not specified 



(continued on the next page) 
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Evaluation 

Characteristics 


Stiefel (1974) 


Bronner & 
Schroder (1983) 


Beywl (1988) 


Nork (1991) & 
Gotz (1998) 


Haring (2001) 


Evaluation Process 


Evaluation through 
the definition of 
criteria derived 
from organisational 
and training 
development 
needs 


Not specified 


Emergent process 

Pre-ordinate 

process 


Transfer to the 
system (transfer of 
learning to the 
workplace) 

Transfer within the 
system (transfer 
within the 
environment of the 
workplace) 


Circular 

Aligned with the 
wider process of 
education 


Evaluation Cycle/ 
Timeframe 


Temporal types of 

evaluation 

• During (the 
learning 
process) 

• After (the 
learning 
success) 

• Follow-up (the 
transfer) 


Not specified 


Not specified 


Evaluation of short- 
term vs. long-term 
changes 


Before & during the 
programme 

After the 
programme 


Evaluation Methods 


Quasi-experimental 
evaluation designs 


(not part of the 
approach) 


Deductive 

(statistical/ 

experimental) 

Inductively 

(qualitative/ 

hermeneutic) 


Evaluation of the 
primary & 
secondary 
elements in the 
didactic field 


Predominately 
qualitative 
interviews and 
survey methods 

Estimations and 
forecasts 


Strengths 


Wider systems 
perspective (linking 
strategy, HRD 
goals and 
evaluation) 


Demonstration of 
the complexity of 
achieving learning 
success 


Classification of the 
theoretical 
foundations of an 
evaluation model 


Emphasis on 
transfer 

Analysis of the 
didactic field 


Comprehensive- 
ness of the 
approach 


Weaknesses 


One way influence 
of strategy to 
education needs 


Closed nature of 
the model 

No organisational 
variables 


(none) 


(none) 


Misleading 
terminology in the 
description of 
stages 



Table 2.12: Summary of the German-speaking Approaches in Psychology/Pedagogy 



As we did before, we raise the question of what we have learned from the German-speaking 
approach in psychology/pedagogy? Thereby, we provide further answers to the fourth 
research question (LS4). 

The selected German contributions from psychology and pedagogy are important for our 
research in two ways. First of all, the different writers call for more comprehensive evaluation 
approaches on the grounds of the inherent complexity of training and development. Bronner 
and Schroder (1983) provide us with a scheme of possible factors that influence the 
learning success. We deduce that many factors come into play and that a clear isolation of 
effects is impossible. In the same way, Nork (1991) and Gotz (1998) contribute other 
variables that come into play when investigating the transfer of learning. Furthermore, Beywl 
confirms that the search for an objective, value-free evaluation is extremely difficult. We 
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understand that our new evaluation approach should be able to handle these difficulties and 
respond to the complexity of the real world. 

The message from these approaches is clear - well-thought conceptional evaluation models 
are ‘nice-to-have’ in theory, but the applicability and usefulness in practice is more critical. 
One should be aware of the multitude of factors and the different aspects of value in the field 
of training and development. Haring (2001) verifies the contributions above by illustrating the 
limited feasibility of approaches. 

Secondly, facing the justified question of ‘So what?’, the German-speaking approaches give 
good advice as well. Stiefel (1974) mentions the importance of linking evaluation to corporate 
goals and objectives. According to “what gets measured gets done" (Patton 1998, p. 230), 
the focus of the evaluation will help aligning the content of education towards the strategic 
direction of the company. Additionally, Beywl (1988) offers classifications of evaluation that 
assists in making decisions with respect to designing our new approach. Finally, Haring 
(2001) lists useful tools for an evaluation approach. 

The German approaches in psychology/pedagogy teach us about the structure of the 
evaluation complexity. They do not provide ready-made evaluation approaches like 
Kirkpatrick or Brinkerhoff, but they help us to think about the problem. A more detailed 
comparison of Anglo-Saxon and German-speaking approaches is found in section 2.3.3. 



2.3.2 Underlying Evaluation Concepts in Finance and Controlling 

“Pedagogical instruments concentrate on the evidence of the learning success, 
whereas economic instruments focus on the behavioural success in the job 
environment and ultimately on the financial measurement of the investment 
success. " 



(Haring 2001, p. 73; translated from German) 

Having illustrated the underlying psychological and pedagogical approaches, we now take a 
look at the fields of finance and controlling. These contribute to evaluation research as well, 
in particular in the form of an analysis of the value or worth of training and development. The 
rationale for including a finance and controlling perspective derives from the overall direction 
of our thesis, which includes the assessment of the value of executive programmes. 
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In contrast to many of the broad evaluation approaches in section 2.3.1, the concepts 
presented here concentrate solely on a specific part of executive education. Their focus is on 
the monetary benefit, the value or worth of training and development, and they thereby lack 
a broader perspective on HRD. Al-Khayyat and Elgamal call these focused approaches 
micro models (1997, p. 89). 30 The analysis of the value of executive education represents a 
sub-function of the wider evaluation research. Therefore, the following section is of a smaller 
scope than the previous section on psychological and pedagogical concepts. 

Like in section 2.3.1, we distinguish between Anglo-Saxon and German-speaking 
approaches as well. In particular, North American researchers have provided a selection of 
mathematical studies of evaluation. Fewer of these can be found in the German-speaking 
literature. Thereby, it seems that Anglo-Saxon writers are more at ease when estimating the 
returns of training and development (cf. Gulpen 1996). German-speaking evaluation 
research has benefited from a stream of thought from the area of controlling. 31 

Specifically, with the following section we answer the fifth research question (LS5) - “What 
are specific efforts to assess the impact -the retum-on-education - of executive education ?" 



2.3.2.1 Approaches in the Anglo-Saxon Literature: The Belief in Measurability 

"... the language of business is dollars, not correlation coefficients ..." 

(Cascio 1982, p. V) 

To date many different approaches have been published in the Anglo-Saxon literature on the 
monetary measurement of the value of human resources (cf. Cascio 1991; Murphy and 
Zanvakili 2000), of management development (cf. Kelly 1993; King 1993) and of training (cf. 
Fitz-Enz 1988; Phillips 1996, 1997). In order to explore the theoretical framework of our 
thesis, we include three Anglo-Saxon approaches in the following subsection, which have 
impacted the literature considerably (cf. Eichenberger 1992). 

The first, and oldest approach is the Utility Analysis, derived from microeconomics, which 
has been pioneered by Schmidt et al. (1982). Secondly, we discuss the ROI Analysis, 



30 Whereas, macro models account for a wider range of internal and external factors in the HRD environment (cf. 
Al-Khayyat & Elgamal1997, p. 89). 

1 The German-speaking evaluation research has coined the term “Bildungscontrolling" (Beywl & Schobert 2000), 
which means ‘controlling of education’. 



73 




derived from finance (Fitz-Enz 1988; Phillips 1994a, 1996c, 2000). The third approach 
illustrates the area of Human Resource Accounting, derived from accounting (Sheppeck & 
Cohen 1985; Cascio 1991). All three approaches demonstrate the aspiration to state the 
impact of corporate education in financial measures and therefore show a certain belief in 
measurability. 

Utility Analysis 

After World War II, American writers started to apply microeconomic calculations to the field 
of personnel decisions - primarily to analyse the impact of hiring decisions. Over the years 
the application base of these calculations broadened and researchers started to use them for 
evaluating training and development as well (cf. Mathieu & Leonard 1987). The theoretical 
development was driven by practitioners’ demand for hard financial data in order to assess 
the value or utility of training and development (cf. Larson 1985; Fischer & Weinberg 1988; 
Birati & Tziner 1999). 

Inspired by microeconomic analysis, Schmidt et at. (1982) devise the underlying formula of 
the Utility Analysis for assessing the dollar value of a training programme: 

AU = T N d t SD y - N C 

Where: 

AU = the change in the dollar value of a training programme, 

T = the duration, in number of years, of a training programme’s effect on performance, 

N = the number of individuals trained, 

d t = the true difference in job performance between the average trained and the average 
untrained employee in units of standard deviation, 

SD y = the standard deviation of job performance in dollars of the untrained group, and 
C = the cost of training per individual 

(Schmidt etal. 1982, p. 335). 

The formula comprises a linear and predominately static approach to the return of training. In 
other words, funds are invested in the black box of training, which generates utility for each 
participant. 

The critical issue of this formula, however, is the measurement - or the estimation - of the 
exact value of the difference in job performance before and after the training d t and the 
standard deviation of the job performance of untrained employees SD y (cf. Mathieu & 
Leonard 1987; Gulpen 1996). Whereas the latter can be produced through a variety of 
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methods 32 (cf. Bobko et al. 1983), the first variable d t is the ‘Achilles heel’ of the utility 
approach (cf. Gulpen 1996). The true increase of performance, which has been caused by 
the training, is often difficult or impossible to measure and must be estimated in many cases. 
Thereby, measurement errors, misinterpretation or bias often distort an ‘objective’ result (in 
section 2.3.4 we discuss the problems and limitations of evaluation further). 

As a result, we observe that the Utility Analysis applies a mathematically ‘objective’ method 
of aggregating the benefits from training. However, the assumptions at its core - i.e. that 
each performance increase can be measured and controlled for - renders the Utility Analysis 
open for ‘subjective’ interpretation. We conclude that mathematical rigour cannot solve the 
question of the value of training and development. 

The original formula of Schmidt et al. (1982) has been developed further to include a time- 
based approach (Boudreau 1983; Mathieu & Leonard 1987), tax and discount rates 
(Eichenberger 1992) as well as job distinctions (Gulpen 1996) and the uncertainty that a 
trained employee leaves the firm (Birati & Tziner 1999). 

The core measurement problem, however, has been left unsolved by all authors. Hence, 
we can consider the Utility Approach useful only in a theoretical way. The abstract formula 
introduces the key variables, which are needed for a value analysis. Yet, it remains very 
abstract and generic. We conclude that a practical implementation of the formula to 
executive education is impossible. 

ROI Analysis 

A second, less formal approach is the analysis of the return-on-investment of training and 
development (cf. Stiefel 1974) - short ROI Analysis. Inspired by Kirkpatrick’s Results level, a 
couple of writers concentrated their research solely on the fourth level (Fitz-Enz 1988; 
Phillips 1991b, 1996c, 1997a+b, 2000; Bramley 1998; Spitzer 1999). 

Fitz-Enz (1988) proposes a conversion of training outcomes into monetary values. He 
describes an approach, which he calls the Training Value Analysis (p. 21). 



32 Gulpen (1996) lists the three most common methods for arriving at SD y : 

(1) Schmidt et al. Method - an estimation of the standard deviation of untrained employees by superiors and 
experts (cf. pp. 46-47), which has proven to be relatively exact, 

(2) CREPID Method (Cascio-Ramos Estimate of Performance in Dollars) - a detailed analysis of the 
components of an employee’s activities (cf. p. 47), 

(3) 40% Rule - a rule-of-thumb estimation, which estimates that the variance of performance accounts for 40 
to 70% of the average salary per year (cf. pp. 47-48). 
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“The concept of value adding implies that there is a positive difference between 
(1) the cost or investment and (2) the benefit or return on that investment. ” 



(Fitz-Enz 1988, p. 17) 



The underlying logic of Fitz-Enz’s Training Value Analysis is similar to the Utility Analysis. 
Assuming that costs can be determined reliably, 33 Fitz-Enz prescribes a linear process for 
calculating the benefit of the training and development (interesting enough most writers see 
the core challenge of evaluation in determining the concrete benefit of training and 
development, however, one of the insights of our thesis has been that assessing costs can 
prove to be an equally difficult procedure - please refer to section 2.3.4). 
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Figure 2.8: T raining Value Analysis 

Source: Fitz-Enz 1988, p. 22 



We acknowledge the underlying logic of Fitz-Enz’s Training Value Analysis. The process of 
generating value becomes clear. However, we consider Fitz-Enz’s concept as simplistic - 
hiding the core measurement problem. The crux of figure 2.8 are the arrows, which are 
difficult to be traced in reality. Fitz-Enz does not explain how ‘benefits’ are attributed to the 
beneficiaries and to the outcomes of training and development. We conclude that Fitz-Enz 
contributes a theoretical guideline to the evaluation debate, but cannot offer practical 
evaluation solutions. 



In comparison to Fitz-Enz, Phillips (1997a+b) becomes more specific in prescribing how to 
calculate the return-on-investment for training and development. Phillips extends 
Kirkpatrick’s model by a fifth level, which he calls ROI level. It remains debatable, whether 
Phillips’ fifth level represents a distinguishable additional level. Rather, we consider a ROI 
analysis as part of Kirkpatrick’s fourth level Results, since the return-on-investment can be 
the result for the organisation. The only purpose that we see in Phillips’ fifth level is the 
execution of a simple calculation. Therefore, we find it difficult to accept the ROI as another 
evaluation level. 



33 Fitz-Enz provides a very detailed List of Cost Items by Phase for a training and development intervention (cf. 
1988, p. 18). 
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Level 


Measurement Focus 


(1) Reaction & Planned Action 


Measures participant satisfaction with the programme 
and captures planned actions. 


(2) Learning 


Measures changes in knowledge, skills and attitudes. 


(3) Job Applications 


Measures changes in on-the-job behaviour. 


(4) Business Results - 


Measures changes in business impact variables. 


(5) Return-On-Investment 


Compares programme benefits to the costs. 



Table 2.13 Performance Measurement 

Source: Phillips 1997a, p. 44 



Phillips suggests different methods to measure level four impact, like follow-up 
questionnaires, programme assignments, action planning, performance contracting, follow- 
up sessions and performance monitoring (cf. Phillips 1997a, p. 46). However, we find that 
these methods primarily contain estimations of the benefit by various parties that are 
involved (cf. Haring 2001). Mainly qualitative data on potential benefits is converted into 
monetary values with the help of participants or experts (cf. Phillips 1994b). 




Figure 2.9: ROI Model 

Source: Phillips 1997b, p. 25 



At the ultimate, fifth level Phillips calculates the return-of-in vestment by using the benefits 
from the fourth level and the costs. 

ROI (%) = [(net programme benefits) / (programme costs)] x 100 
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The ROI approach described by Phillips seems simple at first sight. Phillips applies a 
financial method, which has been used in accounting and finance for decades (cf. Brealey & 
Myers 1996). 34 Phillips’ work contributes to the evaluation debate as it certainly stresses the 
need to work with financial data in the area of human resources (cf. Murphy & Zandvakili 
2000). Yet, taking a closer look we can see that Phillips does not offer a new measurement 
method. Phillips suggests many ways of estimating the benefit of training and development, 
but he remains vague on other methods that do not rely on estimations. He mentions “ trend 
line analysis of performance data ”, “forecasting methods” and “calculating the impact of other 
factors” ( 1997a, p. 48), but does not specify any of them. 

Thereby, Phillips hides the core measurement problem, which we already found in our 
discussion of the Utility Analysis. Consequently, we cannot be sure of the effects of 
education due to the complex array of intervening factors (Thierau 1991). 

Considering relatively technical training, Phillips’ method might entail some success. 
Improvements of time, speed or quality for example can be attributed to manual training and 
subsequently calculated. Bramley (1998) refers to a training example, which has successfully 
implemented a ROI analysis in the style of Phillips’ approach. The example concerns a 
sample of drivers of the UK Post Office Property Holdings, which had caused considerable 
damage by careless driving on duty in the past. A driving training was conducted and 
consequently the benefit was calculated on the basis of the reduced accident damage 
registered. A positive ROI resulted. 

We conclude that Phillips’ approach contains a fairly limited range of application for technical 
and skill-based training. Yet, we find Phillip’s approach unsuited for the evaluation of the 
more abstract executive education of a corporate university. 

Human Resource Accounting 

A third stream of thought from finance and controlling is Human Resource Accounting. The 
Human Resource Accounting approach originated in the 1960s (Lickert 1967; Cascio 1991) 
and can be defined by the following quote. 



Birati & Tziner (1999) contribute the idea of using the Internal Rate of Return instead of ROI. Thereby, the 
decision whether to train or not becomes easier to make for a company. As long as the IRR is higher than the 
corporate cost of capital, training professionals should proceed with the training. 
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“Human Resource Accounting is a method for systematically measuring both the 
asset value of labour and the amount of asset creation that can be attributed to 
personnel activities." 



(Steffy & Maurer 1988, p. 278). 

The approach comprises the application of accounting methods to the human resources, and 
thus also to training and development. At the time, Human Resource Accounting remained 
relatively theoretical and found little practical implementation (cf. Eichenberger 1992). 
However, recently the idea started to flourish again in the framework of Intellectual Capital - 
an offspring of the original approach (Edvinsson & Brunig 2000). 



Cascio (1982, 1991) distinguishes between two methods in Human Resource Accounting, 
i.e. those using a cost approach and those using a value approach (cf. Sheppeck & Cohen 
1985; Lengnick-Hall & Lengnick Hall 1988). 



Cost Approach 


Value Approach 


Historical-cost Method 
Replacement-cost Method 


Stochastic Reward Models 
(Net-Present-Value Analysis) 



Table 2.14: Measurement Methods in Human Resource Accounting 

Source: Derived from Cascio 1982, pp. 59-60 



The cost approach draws on existing accounting methods for valuing assets to be recorded 
in the balance sheet (Flamholtz 1986). Two of the most prominent cost models are the 
historical-cost method (cf . Sheppeck & Cohen 1 985) and the replacement-cost method (cf. 
Eichenberger 1992). 

Translated to the field of evaluating training and development, human resources can 
therefore be regarded as assets, which are valued by their costs parameters. These human 
assets can be valued historically by aggregating the amount of money spent on training and 
development (i.e. historical-cost method). Alternatively, their value can be estimated by the 
amount of money needed to replace a person (i.e. replacement-cost method). Thereby, the 
worth of training and development is evaluated indirectly by the changes in the various cost 
categories. The inherent problem is that these cost approaches are likely to be backward 
orientated and static (cf. Lengnick-Hall & Lengnick-Hall 1988), i.e. the two approaches do not 
consider the future value of the human assets. 

Cascio’s second method in Human Resource Accounting, the value approach, receives 
increasing attention. Hereby, employees and managers are seen as important resources to 
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organisations. In contrast to tangible, material assets, their human capital (Becker 1964) can 
be appreciated by training and development. 35 The approach does not evaluate the input 
costs, but considers the potential future output of the human capital. The accounting 
method uses stochastic reward modeis (Eichenberger 1992), which imply that the value of an 
intervention is analysed and judged by the stream of future contributions resulting from it. 
These future contributions are discounted to a present value using the Net-Present Value 
approach (Brealey & Myers 1996). The approach comprises a dynamic and future-oriented 
evaluation of training and development. 

The value approach emphasises that human capital cannot only be acquired through 
selection and hiring, but also be grown internally by corporate education (cf. Schuler & 
Jackson 1999). Measuring this growth is the prime focus of the value approach. 

The difficulty with both Human Resource Accounting approaches is that no universally 
accepted accounting procedures exist for estimating the various parameters of human 
resources (cf. Sheppeck & Cohen 1985). The existing accounting methods are cumbersome 
and often based on rudimentary estimations. 

We conclude that the value approach of Human Resource Accounting seems the most 
promising Anglo-Saxon contribution in the field of finance and controlling. We appreciate its 
future oriented evaluation perspective. However, in line with the other concepts, the value 
approach cannot offer an applicable methods to measure the financial contribution of training 
and development. With respect to developing our evaluation approach, Human Resource 
Accounting thus contributes a better understanding of ‘ accounting for human capital’, but 
cannot offer any practical advice. 



Summarising the subsection - we provide an overview of the Anglo-Saxon approaches in 
finance and controlling in the following table. 



Evaluation 

Characteristics 


Utility Analysis 


ROI Analysis 


Human Resource 
Accounting 


Evaluation Focus 


Value-oriented 

Value of training and 
development 


Investment-oriented 

Financial return of 
investment (ROI) of training 
and development 


Cost- and investment- 
oriented 

Appreciation of human 
capital 


Type of Evaluation 


Summative 


Summative 


Summative 



(continued on the next page) 



35 Tangible, material assets are only depreciated over time. 
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Evaluation 

Characteristics 


Utility Analysis 


ROI Analysis 


Human Resource 
Accounting 


Level of Analysis 


Individual & organisational 


Organisational 


Organisational 


Evaluator 


Management of corporate 
education 

An economist 


Management of corporate 
education 

A financially versatile 
person 


Management of corporate 
education 

An accountant 


Stakeholders 


Not specified 


Not specified 


Not specified 


Evaluation Process 


Calculation of the value of 
training and development 
through a formula 


Extension of Kirkpatrick’s 
five levels 

ROI analysis 


Accounting for the 
appreciation of human 
capital using either a cost or 
value approach 


Evaluation Cycle/ 
Timeframe 


Variable 

Depends on the lengths of 
the effects 


Evaluation after the 
programme 


Historical analysis (cost 
approach) 

Future oriented analysis 
(value approach) 


Evaluation Methods 


Microeconomic analysis 


Follow-up questionnaires 
Programme assignments 
Action planning 
Performance contracting 
Follow-up sessions 
Performance monitoring 


Cost approach 

- Historical-cost 

- Replacement-cost 
Value approach 

- Stochastic reward 
models (net-present- 
value analysis) 


Strengths 


Outline of variables 
important to the value 
creation of education 


Emphasis on the 
relationship of costs and 
financial benefits 


Asset view of human 
resources 


Weaknesses 


Feasibility 

Problem of estimating the 
influence of the educational 
programme 


Feasibility 

Measurement of the 
financial benefits of 
education 


Measurability 

Estimating the cost/value of 
human capital 



Table 2.15: Summary of the Anglo-Saxon Approaches in Finance and Controlling 



Again we pose the question of what have we learned from the Anglo-Saxon approaches in 
finance and controlling? With respect to our fifth research question (LS5) “ What are specific 
efforts to assess the impact - the return-on-education - of executive education?” we have 
found the following answers. 

The three approaches introduce us to possible financial calculations that can be made in 
the field of evaluating training and development. The Utility Analysis shows us the underlying 
economic structure of the training’s value creation and names the important variables. Even 
though the Utility Analysis cannot solve the ‘return-on-education’ problem, the crux of 
measuring the value of education is highlighted, i.e. how to arrive at the true change in job 
performance. Overall, we consider the utility formula as very helpful to understand and think 
about the measurement problem. However, we realise that in terms of calculating utility in 
real life the formula is far too abstract in our eyes. 
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The ROI Analysis introduces an interesting idea from finance. We learn that other disciplines 
can offer useful tools to the field of evaluation. ROI has been a very popular and powerful 
tool in managing larger businesses, for example decentralised conglomerates with a holding 
structure. Yet, despite the recent attention on ROI Analysis in the evaluation literature, we 
gain theoretical insights again. We see that for a financial analysis we must be able to 
determine the costs and the benefits clearly. Abstract executive education renders both 
difficult. The relationship presented in figure 1 .4 (in our introduction) is confirmed through this 
subsection. The more abstract the content of training and development, the harder it is to 
identify tangible benefits. 

The third contribution of Human Resource Accounting proposes an accounting logic to 
training and development. In particular, the value approach assists us in understanding the 
notion of human capital, i.e. taking an organisational perspective on developing people. We 
learn that evaluation should not take a backward perspective, but should try to focus on the 
future potential output that can be created through training and development. 

We infer that the three approaches add little in terms of practical, real life evaluation 
methods. We would not be able to take any of the Anglo-Saxon approaches in 
finance/controlling and implement it straight into practice. The ‘So what?’ of subsection 
2.3.2. 1 consists of the insight that there is no best way to assess the monetary value of 
executive education. Academics have thought about the problem in theory, but practice 
renders these approaches unfeasible. The principle take-away from this subsection is a 
better understanding of the underlying structure and dynamics of the evaluation problem (we 
elaborate on the main evaluation problems in section 2.3.4). 



2.3.2.2 Approaches in the German-Speaking Literature: The ' Bildungs ■ 
controlling ’ Perspective 

“Evaluation and controlling differ clearly in so far as they originated from different 
traditions of thought ” 



(Pieler 1998a, p. 151; translated from German) 

The German evaluation literature has produced fewer financial approaches in comparison to 
Anglo-Saxon research. However, a growing number of contributions from the field of 
controlling catch the eye (c.f. Riedel 1992; Giilpen 1996; Herget & Beicht 2000). The distinct 
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research area of “Bildungscontrolling” 36 (cf. Wunderer & Sailer 1987a+b; von Landsberg 
1992; Feige 1993; Rank & Wakenhut 1996; Kalmbach & Bracht 1997; Beywl & Schobert 
2000) has evolved next to the traditional psychological/pedagogical stream of evaluation. 
‘Bildungscontrolling’ appears to be a predominantly German approach to evaluating 
education and is therefore dealt with separately in the following subsection of our thesis. 

“Strategic ‘ Bildungscontrolling ' comprises the long-term planning, steering and 
controlling of all educational interventions within an organisation as well as a 
systematic alignment of educational interventions to organisational goals.” 

(Thom & Blunck 1992, p. 38; translated from German) 

The trend of ‘Bildungscontrolling’ is driven foremost by practitioners (cf. Bracht & Kalmbach 
1995; Kalmbach & Bracht 1997) who attempt to apply controlling logic to corporate 
education. In the following subsection, we outline the peculiarities of the ‘Bildungscontrolling’ 
approach by concentrating on its three principal functions. 

Firstly, ‘Bildungscontrolling’ has originated from the areas of controlling and management. 
Therefore, it embodies a planning function. Secondly, the approach includes a focus on 
effectiveness and efficiency. We can therefore discern a steering function. Thirdly, 
‘Bildungscontrolling’ has a controlling function, which is often equated with evaluation 
(Schulte 1992; Feige 1993; Kammel 2000; Pieler 2000). 

Planning Function 

Most evaluation approaches mentioned in previous sections have the objective to improve 
training (formative evaluation) and to determine its benefit or value (summative evaluation). 
Thereby, we have seen that many of the evaluation concepts centre solely on the training 
and development (e.g. Bronner & Schroder 1983). ‘Bildungscontrolling’, however, represents 
a broader managerial approach to corporate education and evaluation. It places training 
and development in the wider organisational perspective and regards education as a 
corporate process among others (Wunderer & Sailer 1987). One of its objectives thereby is 
to plan education. 



36 ‘Bildungscontrolling’ can be translated as ‘controlling of education’. For the remaining discussion we use the 
German word in order to show the distinctiveness of the approach. 
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Brettel (1999) distinguishes between operative planning and strategic planning of 
education. 37 In particular the operative planning function demonstrates the managerial nature 
of ‘Bildungscontrolling’. Immenroth (2000) lists several planning purposes. According to him 
‘Bildungscontrolling’ plans the “demand for education ", the “portfolio of interventions”, the 
“costs, the organisation & realisation” and the “resources” (p. 35). Hence, 

‘Bildungscontrolling’ assists decision-makers in the overall planning of corporate education in 
the company. 

The strategic side to planning implies that the planning of education is integrated into the 
strategy process (Kammel 2000). We observe that the strategic planning function of 
‘Bildungscontrolling’ is similar to the ideas of Stiefel (1974), i.e. educational goals are derived 
from the overall strategy of the organisation. Applying our conceptional framework of SHRD, 
we can see that the strategic planning function of ‘Bildungscontrolling’ inter-links with our 
aligning role of SHRD (see subsection 2. 2. 1.1). Papmehl and Baldin describe the strategic 
planning as “gaining a strategic competitive advantage over competitors through an 
adequate qualification of human resources” ( 1989, p. 815). Consequently, if applied together, 
‘Bildungscontrolling’ and SHRD form a planning and evaluation cycle between corporate 
strategy and education. 

Steering Function 

The second function of ‘Bildungscontrolling’ is closely linked to the aspect of leadership in 
education (Brettel 1999). ‘Bildungscontrolling’ provides the necessary information to support 
“rational leadership” (p. 270), i.e. well-informed decision-making in education. It aims at 
ensuring “transparency of the costs and benefits of education” ( Hummel 1999, p. 28) in order 
to ensure the best possible co-ordination. 

In terms of the steering function, we can find the most interesting contribution of 
‘Bildungscontrolling’. The strong influence from the controlling side (Feige 1993) introduces 
the use of performance measures (indicators) to control for, measure and steer the 
training and development interventions in organisations (Kailer 1996; Herget & Beicht 2000; 
Weiss 2000). 38 The underlying rationale is that these indicators can indirectly demonstrate 
the value of education (Schulte 1992) and thus overcome the difficulties of direct value 
measurement, which we have seen in the finance approaches. For example, the impact of an 



37 Several authors discern an even finer distinction into operative, tactical and strategic (Immenroth 2000). 

38 The use of indicators also forms the basis of the Balanced Scorecard (Kaplan & Norton 1996a), which is 
described in more detail in Section 3.2.2. 
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executive education programme is difficult to assess directly. However, indirectly, indicators 
may demonstrate its value through falling numbers of grievances or decreasing absenteeism 
rates (Kammel 2000). 

“A precise control of the effectiveness of an educational intervention is not 
realistic . Therefore, indicators and qualitative criteria are to be used instead. 
These indirect measurement indicators must be systematically linked to the 
learning and on-the-job application. " 

(Thom & Blunck 1992, p. 44; translated from German) 

In the traditional controlling manner, those indicators are checked regularly and compared 
against a set benchmark or goal (Schulte 1992). Frequently, the basic idea can be compared 
to a plane’s cockpit, where pilots fly the plane by checking a selection of indicators (like 
speed, altitude, geographical location, etc. - cf. Immenroth 2000; Herget & Beicht 2000). 
Wunderer and Sailer (1987a) outline the steering cycle of controlling in figure 2.10. 




Figure 2.10: The Steering System of Controlling 

Source: Wunderer & Sailer 1987a, p. 506 (Translated from German) 



The choice of performance measures and their link to determinants of business success is 
seen as crucial in ‘Bildungscontrolling’ (Weiss 2000). A particular measure should be 
informative and relevant, in order to be used for steering training and development. 
Eichenberger (1992) outlines five standards for choosing indicators to be used in 
‘Bildungscontrolling’. 

First of all, Eichenberger requires the “simple handling (applicability)” of indicators. Secondly, 
he considers it advantageous to ensure a 7ow collection and process costs (cheapness)” of 
the indicators used. Thirdly, Eichenberger advises a “clear organisational and personal 
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allocation (responsibility)" during the collection of indicators, in order to ensure a smooth 
steering process. Fourthly, he requires a “high validity" of the indicators, i.e. the indicators 
should correlate highly with the respective performance or level that they attempt to 
represent. Finally, Eichenberger demands a “high reliability" of indicators so that they remain 
objective across changes in time and different people (cf. p. 94, 1992; translated from 
German). Eichenberger however admits that these requirements are difficult to fulfil in 
practice, and that therefore compromises must be taken. 

As we see in chapter 3, we include the ideas of the steering function and the use of 
indicators of ‘Bildungscontrolling’ in our evaluation approach. Our reasons for using 
indicators derive foremost from the need for a feasible evaluation tool that integrates 
education into the wider framework of the organisation. 

Control Function 

Thirdly, ‘Bildungscontrolling’ has a control function (Pieler 2000). Thereby, we can distinguish 
between the cost control and the benefit control (Brettel 1999). The two types can be 
understood through the notion of efficiency and effectiveness (Feige 1993; Krekel & Gnahs 
2000 ). 



“Effectiveness (goal-achievement-relation ) ... Did we achieve the desired goal?" 

“Efficiency (cost-benefit-relation ) ... Does the benefit achieved by the programme 
exceed the costs caused by it?" 



(Haring 2001, p. 73; translated from German) 

The focus on costs belongs to the traditional purposes of controlling. The control function 
includes the systematic calculation of the costs of education (Kailer 1996) and is considered 
a core function of ‘Bildungscontrolling’ (Herget & Beicht 2000). Gulpen (1996) thereby 
distinguishes between direct and indirect costs, fixed and variable costs, and personnel, 
material and opportunity costs (p. 43). The different types of costs can be measured through 
the traditional controlling methods as for example budgeting or deviation-analysis. Cost 
structures and sum totals are budgeted, checked and analysed per planning period (Thom & 
Blunck 1992). Deviations can be captured through changes in price, amount, consumption 
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and activity (Immenroth 2000). Generally, the determination of costs of education is seen as 
unproblematic in the framework of ‘Bildungscontrolling’ (Brettel 1999). 39 

Today ‘Bildungscontrolling’ places a greater emphasis on the measurement of the benefit of 
education. Thereby, ‘Bildungscontrolling’ defines ‘benefit’ not only in monetary terms, but 
includes qualitative aspects as well. For example, Herget and Beicht (2000) outline different 
dimensions of benefit. 



Focus: Production/Performance 


Focus: Learning/Human Factors 


Focus: Development 


• Productivity 

• Customer orientation 

• Quality 

• Business Processes 


• Competencies 

• Motivation /Employee 
satisfaction 

• Commitment 


• Innovation/Change adaptation 



Table 2.16: Eight Dimensions of Benefit 

Source Herget & Beicht 2000, p. 77 



‘Bildungscontrolling’ distinguishes more systematically than other approaches between 
direct benefits and indirect benefits. Thereby, it focuses primarily on the monetary return of 
education. Direct benefits could be cost reductions, increases in value or prices. Indirect 
benefits could be achieved through quantity increases, time gained or improved quality. 

In terms of the question of measurability, many authors in ‘Bildungscontrolling’ have 
realised the difficulty of quantifying the results of education (in contrast to most Anglo- 
Saxon approaches). 

“Despite extensive research in the field of quantifying the effects of training, 
simple methods, that can be applied securely, do not yet exist. ’’ 

(Brettel 1999, p. 275; translated from German) 

In general, ‘Bildungscontrolling’ combines pedagogical goals of corporate education 
(focusing on trainers, participants, content, duration, budget, methodology) and general 
economic goals (focusing on budget, operating profit, liquidity, cost reduction, productivity) 
(Von Landsberg 1995). The combination implies that the approach attempts to identify 
controllable pedagogical and economic goals, which can be monitored regularly. The most 
discernible difference to other approaches is the operational priority of ‘Bildungscontrolling’. 
As a practical approach, ‘Bildungscontrolling’ is less interested in understanding the 



39 We discuss the problem of determining the costs of education in section 2.3.4. 
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dynamics of education, but aims at seeking the necessary controlling data to make well- 
founded investment decisions in terms of HRD. 

‘Bildungscontrolling’ integrates training and development into the wider processes of the 
organisation and thus takes a more comprehensive view on corporate education. Whereas 
many evaluation approaches try to generate information in order to legitimise the investment 
by focusing on the output of training and development (Brinkerhoff 1988; Phillips 1994; 
Bramley 1999), ‘Bildungscontrolling’ aims at monitoring training and development in order to 
plan and decide. Thereby, it combines the analyses of input and output in a process oriented 
approach. 

However, there exists a principal danger in following a rigorous ‘Bildungscontrolling’ 
approach. Too much controlling and monitoring might lead to dysfunctional behaviour due to 
the creation of an atmosphere of mistrust (cf. Arnold 1996b). 

"... the more exact, differentiated and correct the measurement and evaluation is, 
the more the involved people will feel monitored and the more their initiative and 
innovation will be strangulated. The more vague, generous and abstract the 
planning, steering and controlling, the more space will be given to new solutions, 
initiatives and innovation. It is about the reconciliation of freedom and constraint, 
confidence and control, order and chaos. ” 

(Neuberger 1991, pp. 299-300; translated from German) 

The introduction of controlling mentality and methods to evaluation research certainly has 
many advantages from a management perspective. However, one needs to be aware that 
education deals with human learning and behaviour, which is clearly different from product 
costs and expenses. The integration of the ‘Bildungscontrolling’ approach therefore calls for 
an awareness for the reality of training and development. 

Summarising the ‘Bildungscontrolling’ concept - from a cultural perspective the 
‘Bildungscontrolling’ approach seems to confirm the general picture of a stereotyped German 
mentality, which places a lot of emphasis on precise monitoring and good planning (cf. 
Hofstede 1980, 1991; Trompenaars 1993). However, the clear classifications and the 
controlling mentality advance the evaluation research. We summarise the 

‘Bildungscontrolling’ approach in the following table. 
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Evaluation Characteristics 


‘Bildungscontrolling’ 


Evaluation Focus 


Planning-, steering-, controlling-oriented 
Evaluation as part of a wider controlling approach 


Type of Evaluation 


Summative 


Level of Analysis 


Organisational 


Evaluator 


Higher management 
Directors of corporate education 
Corporate controlling 


Stakeholders 


Not specified 


Evaluation Process 


Gap analysis of indicators and measures 
Comparison of the target with the actual value 


Evaluation Cycle/Timeframe 


Repeated and frequent measurements after the educational intervention 


Evaluation Methods 


Assessment through performance indicators 


Strengths 


Systematic approach of assessing the benefit of education 

Integration of the evaluation of education into the wider organisational setting 

Use of indicators 


Weaknesses 


Primary focus on monetary data 
Danger of over-monitoring 



Table 2.17: Summary of the German-speaking Approach in Finance and Controlling 



Overall, there are several aspects that we can learn from the ‘Bildungscontrolling’ approach. 
First of all, we appreciate the long-term orientation of ‘Bildungscontrolling’, which gives 
evaluation a clear role in the process of attaining corporate goals through education. We 
understand how evaluation can give feedback in the framework of SHRD through its role in 
the planning, steering and control processes of ‘Bildungscontrolling’. Previous evaluation 
approaches have not mentioned this link . 40 

Secondly, we learn about the use of indicators in evaluation. ‘Bildungscontrolling’ provides 
the practical foundation of how to construct a feasible evaluation approach. Consequently, 
we derive our evaluation methods from the ideas of the Balanced Scorecard - a controlling 
tool. We acknowledge that other evaluation approaches have mentioned the use of 
indicators as well, but it is ‘Bildungscontrolling’ that offers the most comprehensive guidance 
on direct and indirect performance measurement. 

A third learning point consists of the distinctions of the different outcomes of education. 
‘Bildungscontrolling’ assists us in perceiving different dimensions through which value is 
created. We find the distinction useful in the design process of our own evaluation approach. 



40 Stiefel (1974) marks the importance of deriving educational goals from corporate strategy, but only in a one- 
way direction. 
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The final question on ‘So what?’ highlights the strengths of ‘Bildungscontrolling’. In terms of 
evaluation methods, ‘Bildungscontrolling’ emphasises the importance of a systematic and 
comprehensive approach in line with the German-speaking contributions from 
psychology/pedagogy. Thereby, ‘Bildungscontrolling’ remains the most practically oriented 
contribution with respect to measurement out of all evaluation models discussed. Its origins 
in controlling seem to ensure a sound practical grounding. 

Summarising part 2.3 on evaluation - we have answered the fourth and fifth research 
questions by reviewing 12 evaluation approaches. Before proceeding to our last research 
question, we compare the four fields of evaluation approaches that we have looked at. 



2.3.3 Comparison of the Four Fields of Evaluation Approaches 

“There is no one appropriate evaluation design ” 



(Noe 1998, p. 150) 

In the two previous sections we have discussed different concepts from the field of 
psychology and pedagogy and from the area of finance and controlling. Furthermore, we 
have reviewed Anglo-Saxon approaches separately from the contributions of the German- 
speaking literature. Our distinctions enable us to compare the four fields, which we have 
thereby defined. 

Before proceeding into more detail, we would like to emphasise that the four fields do not 
present homogenous entities in themselves. On the contrary, all of the four fields include 
different approaches and opinions towards the question of evaluating education. However, 
we were able to observe general trends during our analysis of the literature. We discuss 
these trends with respect to four criteria of comparison. These are (1) underlying 
assumptions of evaluation research, (2) evaluation goals, (3) evaluation process and 
methods and (4) evaluation outcomes and results. We consider each of the criteria in turn. 



2.3.3. 1 Underlying Assumptions of Evaluation Research 

Given the first separating dimension of psychology/pedagogy versus finance/controlling (we 
call the distinction the academic dimension) we can see that the first assumes a broader 



90 




sphere of effects for education than the latter. The field psychology/pedagogy combines the 
individual and the organisational perspective more often than the contributions in 
finance/controlling. Thereby, the psychological/pedagogical approaches capture the 
complexity of education more comprehensively. Changes in human learning and behaviour 
are at the core of most of the psychological/pedagogical approaches. In contrast, the 
financial/controlling models take a more abstract view of training and development, in 
particular the Utility Analysis and the Human Resource Accounting do. These approaches 
seem to have a greater belief in the measurability of the effects of education. 

We agree that the monetary effects of manual/functional training can be measured quite 
precisely, but we consider it very difficult to measure all the monetary effects of behavioural 
training like executive education. Therefore, our advice is to consider the financial/controlling 
approaches with caution. 

The second dimension deals with the differences between the Anglo-Saxon literature and the 
German-speaking literature (we term the distinction the cultural dimension). We find fewer 
clear differences in this dimension. Yet, we discern a slight variance in the perception of the 
importance of proof versus evidence with respect to evaluation. Kirkpatrick states the 
underlying assumption of the Anglo-Saxon approaches. 

“As we look at the evaluation of reaction, learning, behaviour and results, we can 
see that evidence is much easier to obtain than proof. In some cases, proof is 
impractical and almost impossible to get. ” 



(Kirkpatrick in Abernathy 1999, p. 2) 
“Be satisfied with evidence if proof is not possible." 



(Kirkpatrick 1994, p. 68) 

In other words, according to many Anglo-Saxon writers one should take a more practical 
approach as indisputable proof cannot be attained in the evaluation of education (Watson 
1998). 

In contrast, the contributions from the German-speaking literature seem more cautious in 
dealing with proof and evidence. All of the authors that we discussed refer to the “question 
of causality” (Kammel 2000, p. 446) in one form or another. 
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" The determination of a quantifiable benefit/output is difficult in terms of executive 
education due to the missing measurability and the questionable causality. ” 



(Thom & Blunck 1992, p. 42; translated from German) 

Many authors emphasise that the question of causality poses a significant problem in the 
field of evaluation. From our analysis of the evaluation literature we have thus the impression 
that the German-speaking contributions possess a more realistic, but also a more problem- 
centred attitude towards the question of causality. 

Instead of coming to a premature conclusion, we can formulate only a broad impression. 
Anglo-Saxon authors realise the difficulties of proving the value of education and thus adopt 
practical and uncomplicated measurement procedures. German-speaking authors remain 
suspicious of measuring the financial return of education and thus place a lot of importance 
on proof. 



2.3.3.2 Evaluation Goals 

The second criterion entails the comparison of evaluation goals. In terms of the academic 
dimension, the difference in goals is relatively distinct. The psychological/pedagogical 
approaches are interested in the individual and organisational results of education. These 
results include the qualitative as well as quantitative benefit of individual learning and 
behaviour for the organisation. Financial/controlling approaches have a more limited 
evaluation focus. They are primarily interested in the monetary benefit of education, i.e. the 
quantitative pay-off of the investment in education. 

Looking at the cultural dimension, the differences become blurred and consequently we can 
only give an impression again. Anglo-Saxon approaches started early to investigate the 
bottom line of education, e.g. Kirkpatrick’s model dates from 1959/60. The financial emphasis 
can be seen even in his model, which stems from psychology/pedagogy. Until today, the 
Anglo-Saxon approaches have remained results-orientated in the finance/controlling side 
as well as in the psychology/pedagogy side. 

The German-speaking approaches seem less focused on results, but include a strong 
process view in order to ensure learning success. Thereby, the German-speaking authors 
concentrate on those factors that facilitate learning and the transfer from learning. It seems 
they are more interested in the effective and efficient way of achieving educational results. 
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German-speaking approaches, as for example ‘Bildungscontrolling’, do not avoid the details 
of education and therefore seek a deeper understanding of the underlying dynamics. 
These approaches hence aim at providing a greater transparency of the processes and 
benefits of education. 

Our overall impression therefore is that the Anglo-Saxon approaches follow a clear results- 
oriented objective, whereas the German-speaking contributions are also interested in results, 
but include the aim to understand and optimise the processes of education as well. 



2.3.3.3 Evaluation Process and Methods 

With respect to the third criterion, the differences in the academic dimension derive from the 
varying evaluation goals. We can find a selection of different evaluation processes. Most 
evaluation processes of the psychology/pedagogy side are integrated into the overall cycle of 
education and therefore suggest a continuous activity. Many approaches thereby consider 
the wider system of education in their evaluation process. In terms of evaluation 
instruments, the psychology/pedagogy side includes a broad spectrum of qualitative and 
quantitative social research methods. The wealth of proposed data collection tools and forms 
of analysis (in theory!) is striking. 

In contrast, the financial/controlling contributions are more limited in their processes and 
methods. Many approaches view the evaluation of education as a singular process, which 
has a clearly defined start (before the programme) and end (after the programme). Only in 
some cases are the aftermath effects of education analysed. Furthermore, we observe an 
emphasis on quantitative methods. In particular, the theoretical models, as for example Utility 
Analysis, rely on quantifiable data. The more practical approaches, like ‘Bildungscontrolling’, 
include also a selection of qualitative methods. 

Changing dimensions to the cultural difference, we can discern a transfer of ideas from 
Anglo-Saxon research to German-speaking evaluation research. The model of Kirkpatrick 
serves as the best example of the Anglo-Saxon influence. The model can be found as a 
foundation or starting point in many German-speaking approaches (Nork 1989; Brettel 1999; 
Kammel 2000; Haring 2001). Many other evaluation ideas stem originally from the Anglo- 
Saxon literature and are subsequently introduced into the German-speaking research 
(Eichenberger 1992). In many cases though, the original Anglo-Saxon approach is then 
applied more rigorously by German-speaking authors (Gulpen 1996). The only exception is 
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the area of ‘Bildungscontrolling’. The strong controlling tradition in Germany has generated 
its own understanding of evaluation. It will be interesting to see whether these ideas will 
reverse the direction of influence for once, i.e. whether ‘Bildungscontrolling’ will have an 
influence on Anglo-Saxon research. 

Overall, we can observe more open methods in the field of psychology/pedagogy. The 
analysis of the value of education is often pursued with rather narrow evaluation methods by 
the financial/controlling approaches. Secondly, many of the evaluation ideas originate in the 
Anglo-Saxon world and are subsequently taken up in the German-speaking research. 



2.3. 3.4 Evaluation Outcomes and Results 

Before proceeding with our comparison, we find it interesting, that very few approaches, in 
general, produce tangible evaluation results. All four fields in the literature are 
characterised by predominately theoretical approaches (Pieler 1998; Thierau-Brunner et al. 
1999). Many authors contribute their best-way-solution to the literature and provide many 
possible evaluation methods in theory, but few back-up their approaches with practical 
experiences (Haring 2001). Therefore, we distinguish evaluation outcomes, i.e. broader 
theoretical contributions to the evaluation research, and evaluation results, i.e. specific 
empirical findings. 

Starting with the academic dimension again, the concepts of psychology/pedagogy 
contribute a broader and better understanding of evaluating the effects of education. In 
general the approaches provide structure and transparency to evaluation research. The 
contributions outline the complexity of individual learning and behaviour. Furthermore, they 
suggest differentiated measurement methods, as for example different levels or processes, 
and emphasise important aspects of evaluation, as for example the focus on transfer or the 
systems perspective. 

The finance/controlling side introduces foremost management methods to the field of 
education. These approaches succeed with their quantitative measurement methods in terms 
of manual/functional training (Bramley 1999), but with respect to the more abstract field of 
executive education they have not found a reliable and standardised way to calculate the 
benefit (Brettel 1999). Nevertheless, the financial/controlling approaches enhance the 
understanding of the quantitative side of education as well. 
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In terms of the cultural perspective we are unable to identify any major differences in the 
results and outcomes of evaluations. Both sides reach similar conclusions in terms of the 
evaluation of education. 

Summarising the subsection - we conclude that there are observable differences in terms of 
the academic dimensions. We explain these differences on the grounds of the different 
origins and objectives of the disciplines. However, we do not discern clear differences with 
respect to the cultural dimension. Although, we have the impression that the two camps 
handle the aspect of measurability differently, we do not have sufficient supporting evidence. 
Therefore, we suggest further research in the comparison of the four different camps of 
evaluation research mentioned. 



2.3.4 Measurement Problems in Evaluation 

“A tight chain of cause and effect between educational success and corporate 
success cannot be shown. ” 



(Thom & Blunck 1992, p. 40; translated from German) 

The discussion above shows that the different evaluation models and approaches tackle the 
question of evaluating training and development in their own distinctive manner. However, 
most of these approaches arrive at the same evaluation problems. In the following 
subsection, we discuss the four most common difficulties, which we identified in our literature 
research (cf. Nork 1989; Eichenberger 1992; Riedel 1992; Weiss 2000). Thereby, we 
address the last research question of our Theoretical Frame (LS6), “What are current 
problems and limitations in evaluation research?”. 

The first and most important problem is the difficulty to understand the cause and effect 
between training and development and its outcomes. Secondly, evaluation often faces a lack 
of appropriate data for assessing the impact of training and development. Thirdly, it is difficult 
to quantify success in training and development in monetary terms. And fourthly, temporal 
distortions are likely to influence evaluation results. We deal with these four principal 
problems in turn. 
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2.3.4.1 The Unclear Link of Cause and Effect 



Management training and development takes place in a highly complex organisational 
environment, which includes many intervening variables that are hard to account for (Arnold 
1996). Furthermore, training and development addresses principally cognitive and 
behavioural aspects of the transfer and individual learning (cf. Royer 1979). Thereby, a 
further complexity is added to the evaluation research. We conclude that it is unlikely that the 
relationship between a training and development intervention is a linear and static one. 41 
Rather it can be assumed, that a variety of variables in changing constellations cause 
different effects. The cause and effect structure are thus likely to be multidimensional and 
dynamic. Therefore, it is very difficult to establish a clear cause and effect relationship 
between training and the creation of value for an organisation (Olian et ai. 1998; Garavan et 
al. 1999b). Causalities, which exist in theory, have not been proven through empirical 
evidence (Weiss 2000). Noise of intervening variables make it difficult to trace back the 
effects unequivocally to a particular training and development intervention (Riedel 1992). 
Even experimental or quasi-experimental evaluation designs are unlikely to comprise the 
amount and complexities of variables (cf. Nork 1989). 

Among the evaluation models above, the psychological and pedagogical approaches have 
recognised and accepted this complex state of the world, whereas approaches from finance 
and controlling often still work on the belief of measurability. Arnold (1996b) therefore 
emphasises the “Reflexionsvorsprung’* 2 of the psychological and educational sciences in 
comparison with finance and controlling. In his view, the results of training and development 
are always dependent on the subject, the situation and the context and therefore 
measurement is very difficult (p. 259). Realised or denied, in our view the complexity of 
training and learning render most ROI evaluations very difficult. 

Finally, there is uncertainty of the nature of a possible cause and effect relationship 
(Thomson et al. 1997). The studies mentioned above assume that an increase in the training 
and development investment causes an increase in effectiveness and therefore a rise in 
revenues. At the same time it is quite likely that an inverse relationship exists. Very 
profitable firms are able, and possibly also willed, to spend more on training and 
development compared to less successful firms (cf. Hendry et al. 1989; Horwitz 1999). A 



Linear in this context would imply, that only one input variable would cause a specific output variable to change 
in one direction, i.e. a high degree of correlation exists. 

2 The ‘superiority through the ability to reflect on reality' (Translated from German). 
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famous example is IBM, an organisation that had become very profitable throughout the 
1960s and 70s. The amount of training and development given to employees was considered 
to be a benchmark in the industry. However, with a period of faltering financial success in the 
late 1980s the amount of training and development was also reduced. We can see that 
profitability influenced the amount of training. Therefore, it might be quite ambiguous to 
conclude a one way correlation from training and development to profitability. Reverse 
correlations may apply in the case of very profitable firms that want to share the financial 
success with their employees in the form of training and development. 



2.3.4.2 The Lack of Appropriate Data 

A more practical problem for the evaluation of training and development is the lack of 
appropriate measurement data (Riedel 1992; Weiss 2000). We refer to data on the input, 
throughput (Eichenberger 1992) and especially the output of training and development. 

Eichenberger distinguishes between input (i.e. the cost side), output (i.e. the benefits) and 
throughput. Thereby, he defines throughput as the resources used for analysing the demand 
for training and for developing the training plan or concept (p. 80). Accountants use the 
notion of sunk costs (Brealey & Myers 1996). These costs, which are generated prior to 
training and development, are relatively easy to determine. However, they often account only 
for a small fraction of the total costs. 

In most cases very rough estimations exist on the input side. The most simple way is to 
aggregate the respective direct and indirect costs of training and development (Gulpen 
1996). However, going into more detail, the question opens up, whether the true costs, i.e. 
the true size of the investment can be measured. 

The following illustration by van Adelsberg shows that often only the tip of the cost iceberg is 
seen and that the larger share of costs is of a hidden nature (cf. figure 2.11 on the next 
page). 

The costs that cannot be seen or calculated easily are the lost productivity and lost 
opportunity , which arise through the participation in a programme away from the job. 
Furthermore, van Adelsberg (1999) emphasises the size of the wasted training investments 
that can arise through wrong decision in executive education. 
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Figure 2.11: The Costs of Training 

Source: Van Adelsberg, 1999, p. 52 



We consider van Adelsberg’s cost pyramid as useful since it illustrates the difficulty of finding 
true costs. Yet, we understand van Adelsberg’s pyramid as an abstract rather than a actual 
cost representation. His message is that the hidden costs are likely to represent a 
substantially larger share than the visible costs. We conclude that the actual proportions 
depend on the specific characteristics of executive education and therefore the pyramid 
might look different from case to case (in some cases it might not even have the shape of a 
pyramid). 

We see that assessing the size of the training and development investment, i.e. total costs, 
might not be such a straightforward task after all. There is no objective formula for measuring 
the input (cf. Weiss 2000) and it is unclear which data to include or exclude. 

A further important crux is the measurement of the output side. General information on 
overall sales, market share and net revenues are obtainable in almost all private companies. 
However, in many cases this information cannot be used for measuring the value of training 
and development as it is collected and calculated under a different organisational focus 
(Weiss 2000). General corporate financial information is of a different nature to the kind of 
data needed for assessing the value of training and development. For example, net revenues 
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are calculated according to accounting conventions like US GAAP. Thereby, net revenues 
are not determined through individual contributions of executives, but rather by looking at 
overall sales and subtracting variable and fixed costs and other items like depreciation. 

Combining the measurement problems at the input as well as at the output side, we can see 
that the calculation of the return-on-investment of executive education faces a significant 
problem due to the lack of appropriate data. 



2.3.4.3 The Difficulty to Quantify Effects in Monetary Terms 

Thirdly, the search for the value of training and development encounters the problem of 
translating the effects into monetary terms (Riedel 1992). Training and development often 
address the transfer of cognitive and behavioural learning to the workplace (Gotz 1998). 
Even if changes in attitudes and behaviour were subsequently measured, the impact would 
in most cases be recorded in soft data. Phillips (1994) therefore suggests to convert the soft 
data into hard, financial data through estimations and experts’ opinions (p. 17). However, 
this procedure of converting data leaves considerable room for ambiguity and subjectivity. 
For example, a training and development programme is successful and the participants 
return to their jobs and apply a newly learned strategy tool. It can be proven that their 
decisions in strategy have improved and a relatively clear link of cause and effect can be 
established. Still it remains difficult to derive monetary conclusions on the value created from 
the change in behaviour. 

Overall, we observe that the conversion problem does not concern the evaluation of 
education alone, but represents a more universal quantification problem in organisations. 



2.3.4.4 The Distortion through Temporal Effects 

The last measurement problem to be considered here deals with the issue of delayed 
effects of training and development (Weiss 2000). A training and development intervention 
has no immediate impact structure as for example an investment in new machinery. In most 
cases the value will be generated in a disruptive and delayed nature after the programme. 
For example the theories of the transfer of learning comprise various reasons for triggering 
the application of learning (Cheetham & Chivers 2001). Learning transfer and application 
therefore might take place due to a common stimulus at the work-place. Or it might be the 
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result of a successful memory search process (cf. Royer 1979). For the measurement of the 
value of training and development, this implies that many effects cannot be expected at a 
pre-determinable point of time. For some participants they might materialise within the first 
ten days after returning from the programme. However, in the case of management training 
and development it is likely that the impact is noticed only in the long-run, i.e. weeks or 
months, maybe years afterwards. 

Concluding this section on measurement problems - the four points discussed certainly do 
not cover all difficulties, but they touch on the most common measurement problems and 
give thus an answer to LS6. In particular, the first problem mentioned - the unclear link of 
cause and effect - represents an umbrella for many similar obstacles (Arnold 1996b). Most 
evaluation approaches have not yet found a proven way to measure the exact value. 



2.4 Integration of Content and Method: The Research Frame 

“Executive development as a component of Strategic Human Resource 
Development has emerged as an important lever for gaining competitive 
advantage. ” 



(Horwitz 1999, p. 185) 

In the preceding parts, we have explored the two broad areas of our thesis. We have defined 
our content foundation as building strategy process capability through customised 
education and we have filled the terms with meaning. Subsequently, we have reviewed the 
field of evaluation, which represents the methodical approach of our research. We can now 
integrate content and method in order to create the overall frame for our conceptional 
contribution and our empirical research (cf. figure 2.12 on the next page). 

In the Theoretical Frame we have provided the reader with a specific definition of strategy 
process capability. Thereby, we have overcome the predominately abstract nature of the 
strategy literature. Many authors elaborate on the process of strategy, but few include details 
on the concrete capability required to make and implement strategy (Schwuchow 1992). We 
have pointed out that these strategic skills, knowledge and mentality can be crucial to 
effective and successful strategy-making and implementation. Through our discussion in the 
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Theoretical Frame we have ensured a better understanding of the underlying elements of 
these skills, knowledge and mentality. 




Figure 2.12: Integrating Content and Method 



Additionally, we have shown the novelty of the idea of customising executive education 
through a close partnership of business school and corporation. We provided a definition of 
customisation and elaborated the advantages and disadvantages. Yet, we concluded that a 
clear need for empirical research exists. 

As a result of the content foundation, we have prepared the way for our empirical research 
in later chapters, where we investigate, whether strategy process capability can be 
developed by customised executive programmes (cf. research intention in chapter 1). 

Furthermore, we have used the discussion of the different evaluation approaches as a 
foundation for the development of our new evaluation approach. In the Theoretical Frame we 
have reviewed the merits and problems of existing evaluation concepts in order to learn from 
them and to gain inspiration from them. The development of our new evaluation approach in 
the next chapter thus represents our answer to the challenges of evaluation research. 

Overall, figure 2.12 represents a kind of road-map for the remaining chapters. First of all, 
chapter 3 describes the development of our new evaluation approach. Subsequently we 
apply our tool to a real life environment in chapter 4, which forms the empirical part of our 
research. In chapter 5, we present our results. The practical problems of the implementation 
are discussed in chapter 6. We close with an outlook in chapter 7. 
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3 New Evaluation Approach 



“The traditional understanding of evaluation implying perfect control and 
controllability of success is replaced by a holistic, integrative approach of human 
resource development, corporate education and educational success. Thereby, 
evaluation becomes an integrative part of the process of human resource 
development. ” 



(Epple 2000, p. 2; translated from German) 

An important rationale for the research presented here has been the lack of practical 
evaluation methodologies. Therefore, the primary objective of our thesis has been to 
develop an evaluation tool that can be applied to executive education in real life. In the 
following chapter we describe a new evaluation concept, which we have developed in the 
framework of this thesis. In order to do so, we take a look at some important requirements 
concerning an evaluation tool first. Secondly, we provide an overview of the conceptional 
sources that have contributed to the design and the development of our new evaluation tool. 
Thirdly, we outline the testing and validating of the tool. Then we describe the new evaluation 
approach in detail. Subsequently, we illustrate how the new approach can be applied in 
practice. We finish the chapter by considering the expected benefits that the tool can deliver 
to an evaluator and his/her organisation. 



3.1 The Strategic Tool for Evaluating Educational Returns (STEER) 

“In many instances a more integrative approach has greater benefits. ” 

(Smith 1993, p. 21) 

The literature research has illustrated that there is ‘no one best way’ to evaluate executive 
education. Several different streams of research face the same underlying problems (see 
section 2.3.4) and cannot offer a superior answer to the question of evaluation. Many 
evaluation concepts remain foremost theoretical and lack the practical testing in a real-life 
environment. Therefore, the development and the subsequent application of the new 
approach represents our contribution to the global evaluation discussion. Being aware that 
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the approach presented here is not a panacea to all evaluation problems, we are 
nevertheless confident that it proves valuable to academics as well as practitioners. 

The name of our evaluation approach is The Strategic Tool for Evaluating Educational 
Returns and is abbreviated STEER. The analogy towards the verb to steer is intentional. The 
tool is meant to help steering the resources of executive education towards an optimum 
through evaluation. In line with the results of the Theoretical Frame, the tool’s general 
purpose can thus be formulated as: 

To make the effects of executive education programmes more transparent with 
respect to their value contribution to individual and organisational capabilities. 43 

The approach is designed to suit practitioners in evaluation research in particular. Following 
Patton’s (1997) emphasis on utilisation, STEER has a strong focus on real-life application. 



3.2 Development of STEER 

“The research method and the type of questions asked, ethical problems as well 
as the available time and the costs are crucial for an evaluation. ” 

(Gulpen 1996, p. 16; translated from German) 

The following part on the development of STEER should assist us in tracing back the 
conceptional construction of the new evaluation tool. We outline the important 
requirements of the tool as well as the conceptional origins of it. 

Section 3.2.1 consists of presenting the conceptional requirements of the tool. Thereby, 
the evaluation tool faces conflicting demands. On the one hand, the tool must qualify from 
a theoretical perspective in order to achieve valid and reliable results. On the other hand, the 
evaluation takes place in a real-life environment (as opposed to a laboratory) and thus the 
tool should not be too cumbersome. We illustrate the different requirements of an evaluation 
approach by reviewing four important standards issued by the Joint Committee on Standards 



43 In our empirical part we apply the tool specifically to the aspect of developing strategy process capability 
through customised executive programmes. 
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for Educational Evaluation (Sanders 1994). These standards have been the guideline in the 
development process of the new approach. 

In section 3.2.2 we review the two most influential ideas, which served as the conceptional 
sources of the new tool, STEER. The underlying structure of the tool has been derived from 
the balanced scorecard (Kaplan & Norton 1992). This includes, foremost, the emphasis on a 
balanced assessment through different perspectives. Additionally, the measurement system 
- consisting of indicators - has been derived from the balanced scorecard. The content of 
STEER refers to the Four Levels of Kirkpatrick (1959/60). Here, the idea of different levels of 
value creation are regarded as essential. In addition, the recognition to use different 
measurement methods at different levels has been derived from the Kirkpatrick framework as 
well. 

The last step in the development process has been the pilot testing of STEER. In section 
3.2.3, we give a short overview of the testing and validation procedures undertaken. 



3.2.1 Conceptional Requirements 

Since 1975 the Joint Committee on Standards for Educational Evaluation has assisted 
practitioners in the evaluation of educational programmes. Their goal is to provide a guideline 
for ensuring useful, feasible, ethical and sound evaluation results. The committee has issued 
30 programme evaluation standards, which are classified under four headings. The four 
headings are utility standards, feasibility standards, propriety standards and accuracy 
standards. The four sets of standards are introduced briefly in the following section. For a 
more detailed overview of the contents of all standards, please refer to appendix 8.9. 

We interpret the standards as a description of ‘What makes a good evaluation tool?’. The 
content of the various standards has therefore assisted us in defining the conceptional 
requirements of STEER. 



3.2.1. 1 Utility Standards 

“The utility standards are intended to ensure that an evaluation will serve the 
information needs of the intended users. ” 

(The Joint Committee on Standards for Educational Evaluation, Sanders 1994, p. 23) 
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The utility standards are closely linked to the ideas of Patton (1997). They require that the 
evaluation methodology is informative, timely and influential. The evaluator should 
acquaint himself/herself with his/her audience first and define the important stakeholders 
clearly. Furthermore, the evaluator should ascertain the stakeholders’ information needs, in 
order to respond to these needs and to report the relevant information clearly and in a timely 
fashion. (Sanders 1994, p. 23). 

In the framework of STEER, we define the staff responsible for executive education as its 
intended primary users. These users might be regular employees of corporate universities, 
key decision-makers in Strategic Human Resource Development areas or assigned 
evaluation specialists in the field of Human Resources. We consider that information 
demands are likely to differ among these intended users (Cascio 1991). The variation in 
information demand is likely to be influenced by the level and function of users. Higher level 
decision-makers are likely to seek a general overview, whereas specialists require detailed 
data. Likewise a controller within the training and development area has a different 
information need compared to a professional trainer. 

We developed STEER for actual and repeated use in the real-life setting of executive 
education. In line with the utility standards, we involved the intended users in the design of 
the tool (cf. section 3.2.3). At the end of the day the evaluation results of STEER are meant 
to guide decisions and choices in executive education. 



3.2.1. 2 Feasibility Standards 

“The feasibility standards are intended to ensure that an evaluation will be 
realistic, prudent, diplomatic, and frugal. ” 

(The Joint Committee on Standards for Educational Evaluation, Sanders 1994, p. 63) 

One of the most crucial requirements of the new evaluation tool concerns its applicability. 
The past has shown that theoretically superior designs are frequently of little value because 
they are not practicable. In terms of the practical design of the evaluation approach, we 
considered a number of feasibility factors. 

First of all, the evaluation tool should be operable from a technical point of view (Sanders 
1994, p. 65). The evaluator, i.e. most likely the staff of executive education, must possess 
the technical capacity to apply the methods of the evaluation. For example, an evaluation 
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technique that requires measuring the neurological activity of participants’ brains is clearly 
inappropriate for regular corporate education since the staff does not possess the technology 
required. 

Secondly, the evaluation should be realisable from an organisational point of view. In 
particular, the organisational setting might place many time and availability constraints on the 
evaluation. For example, senior executives have very demanding work loads and schedules. 
Hence, it seems unlikely that these high level managers are available for extended periods of 
testing and interviewing. 

Thirdly, the method should be accomplishable by the evaluating staff. The chosen 
methodology should match the skills of the people using it. For example, it is unlikely that 
regular corporate university staff are able to conduct highly sophisticated statistical analyses. 

Fourthly, the application of the tool should be reasonable in terms of invested resources. 
All evaluation methodologies should be assessed from a cost-benefit perspective. For 
example, assigning an evaluator to each participant - for close after-programme 
observations - seems too expensive for an organisation in terms of salary costs. 

Lastly, there are political constraints, which are often underestimated (Sanders 1994, p. 71). 
An evaluation tool should be feasible politically. For example, an evaluation approach might 
fail since it tries to combine conflicting interests. Different people have different stakes in the 
evaluation of executive education (Neuberger 1991). Conflicting interests might cause 
dysfunctional behaviour among the staff of executive education in terms of handling an 
evaluation. Data could be manipulated or deleted to avoid negative reviews. Therefore, the 
evaluation tool should be aware of the political repercussions of its focus or design. 

In the development of our evaluation approach, we found the feasibility standards very 
helpful. The standards demonstrate, that evaluation studies in a real-life environment face 
additional complexities as compared to controlled laboratory experiments. The standards 
advise us the evaluations should be realistic, prudent, diplomatic and economical. Therefore, 
we have tried to design our tool in order to handle these organisational complexities and to 
ensure a feasible application. The application of STEER in chapter 4 provides us with 
feedback on whether we have achieved applicability with our design. 
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3.2.1. 3 Propriety Standards 



"The propriety standards are intended to ensure that an evaluation will be 
conducted legally, ethically, and with due regard for the welfare of those involved 
in the evaluation, as well as affected by its results. ” 

(The Joint Committee on Standards for Educational Evaluation, Sanders 1994, p. 81) 

Propriety standards comprise the dimensions of correctness, fairness and objectivity, which 
are important since the evaluation of executive education includes judgements of human 
behaviour. The results of an evaluation are likely to affect different people in a variety of 
ways. An evaluation tool should therefore consider the propriety standards in the following 
aspects. 

An evaluation approach should attempt to follow an ethically correct evaluation procedure, 
i.e. the approach should collect its information objectively and keep a fair judgement 
(Sanders 1994, p. 105). This implies that an evaluation should be free of personal values and 
opinions. In this context Beywl speaks of a value-free evaluation (1988). However, Beywl 
points out that most evaluations are value-laden due to the goals of the stakeholders 
involved. We can see that objectivity is a notoriously difficult concept and that there has been 
no overall valid set of criteria, that guarantee objectivity in a complex and dynamic 
organisational environment (Gulpen 1996). With STEER we have constructed an approach, 
which seeks objectivity through the definitions of different evaluation perspectives (thus not 
following one singular perspective). 

According to the standards, the evaluation tool requires an open evaluation culture as well. 
This implies on the one hand, that data is not distorted or forged on purpose. Referring back 
to the political implications of evaluation, it seems likely that there is often an interest in 
hiding bad training outcomes in order to avoid repercussions. On the other hand, evaluation 
requires an open feedback environment. In particular negative evaluation results should be 
accepted and taken as a motivation to do better and to seek positive change. 

Finally, proprietary standards include the respect of individual rights and legal concerns. For 
example, many German middle managers are part of union agreements. Any survey or 
questionnaire with middle managers must therefore conform to the union’s requirements for 
the protection of the employees. 
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The propriety standards are probably the most difficult to respect because they focus on 
ethical aspects and human behaviour. We have taken the propriety standards as a guideline 
in the development process of our tool, but only the practical implementation will 
demonstrate to what extent we have succeeded. 



3.2.1. 4 Accuracy Standards 

“ The accuracy standards are intended to ensure that an evaluation will reveal and 
convey technically adequate information about the features that determine worth 
or merit of the programme being evaluated. ” 

(The Joint Committee on Standards for Educational Evaluation, Sanders 1994, p. 125) 

The fourth group of standards aims at ensuring that an evaluation produces sound 
information on the value of a programme. The generated results must be comprehensive , 
reliable and exact (p. 126, 1994). As much as the previous standards, accuracy is critical for 
the success of the evaluation tool for several reasons. 

Considering the evaluation as a measurement process, accuracy is clearly desired in terms 
of the quality of results. The generated information is likely to be used for discussions and 
decisions. Thereby, the degree of accuracy influences the quality of decisions. Valid and 
systematic information enables decision-makers to arrive at a sound judgement of training 
and development. Incomplete and wrong data can create a misleading picture, which is likely 
to induce arbitrary decisions. 

Furthermore, accurate results are recognised and therefore enhance the credibility of the 
evaluation methodology. Credibility is key to the successful continuation of the evaluation 
approach. Besides, credibility is important for the communication of results as well. Results 
that nobody believes in because of their inaccuracy are useless. 

The accuracy standards include good documentation of programme results in order to 
ensure that details can be traced back. Evaluation procedures and sources should be sound 
and well documented. 

However, as noted above, accuracy often conflicts with feasibility. In order to improve the 
accuracy of an evaluation, additional methods and measures must be introduced. The 
additional methods are likely to render the evaluation as a whole more complex and difficult, 
i.e. less feasible. Thereby, considerable costs can often be incurred to increase the degree of 
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accuracy. We see that these evaluation standards cannot prevent common research 
dilemmas. Having outlined the characteristics of an appropriate evaluation tool, we now take 
a look at the conceptional sources of STEER. 



3.2.2 Conceptional Sources 

There are two conceptional sources, which exerted the most important influence on the new 
tool, STEER. Both have provided solutions to the theoretical and practical requirements 
mentioned above. The first one is Kaplan and Norton’s balanced scorecard and the second 
one is Kirkpatrick’s four level evaluation model, which has been introduced in the Theoretical 
Frame. 



3. 2. 2.1 Kaplan and Norton’s Balanced Scorecard 

“No single measure can provide a clear performance target or focus attention on 
the critical areas of the business. ” 



(Kaplan & Norton 1992, p. 71) 

In 1992 Kaplan and Norton started publishing a series of articles (Kaplan & Norton 1992, 
1993, 1996b; Norton 2000), which emphasised the importance of non-financial performance 
measures. Kaplan and Norton thereby proposed a new management tool that complemented 
the prevailing strong financial management control in North American corporations at the 
time (Rudolph 1994). 

“The balanced scorecard supplements traditional financial measures with criteria 
that measures performance from three additional perspectives - those of 
customers, internal business processes, and learning and growth. It therefore 
enables companies to track financial results while simultaneously monitoring 
progress in building the capabilities and acquiring the intangible assets they 
would need for future growth. ” 



(Kaplan & Norton 1996b, p. 75)“ 



44 The text was originally written in the past tense (cf. Kaplan & Norton 1996b, p. 75). To suit the flow of the 
description of the Balanced Scorecard, we quote the text in the present tense. 
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Kaplan and Norton's balanced scorecard derives from earlier concepts like the French 
Tableaux de Bord (Epstein & Manzoni 1998). The balanced scorecard takes the form of a 
comprehensive overview of key performance indicators of the business. Kaplan and Norton 
have compared it to “the dials and indicators in an airplane cockpit" (1992, p. 72). The 
scorecard is centred around the vision and strategy of the organisation, which represents its 
core. From the vision and strategy four sets of indicators are derived according to four 
different perspectives, the financial, the internal business processes, the learning and growth 
and the customer perspective. Each perspective includes indicators and measures that 
monitor the success of the business as for example market share in the customer 
perspective. 



"To achieve 
our vision, 
how should 
we appear to 
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customers?" 


Customer 
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Financial 


Objectives Measures Targets Initiatives 




























"To satisfy 
our 

shareholders 

and 

customers, 
what 
business 
processes 
must we 
excel at?" 



Internal Business Processes 


Objectives Measures Targets Initiatives 



















"To achieve 
our vision, 
how will we 
sustain our 
ability to 
change and 
improve?" 



Learning and Growth 


Objectives Measures Targets Initiatives 











Figure 3.1 : The Four Perspectives of the Balanced Scorecard 

Source: Kaplan & Norton 1996, p. 76 



The balanced scorecard prescribes an iterative measurement process. After the four sets of 
indicators have been derived from the vision and strategy, each indicator is defined 
specifically. The objectives of the indicator are articulated and its target levels are set. 
Subsequently, different measurement instruments are put into place, which measure the 
respective indicators in regular intervals. In the analysis the actual values are compared to 
their targets. The results of the analysis provide a comprehensive overview about the current 
state of the business - confirming those areas, which meet their targets and highlighting the 
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problem fields, which are falling behind their targets. The analysis enables management to 
take specific and fine-tuned action to bring the organisation back on track. Finally, the results 
are fed back into the vision and strategy, and the consequences for the formulation of the 
indicators are considered anew. The adjustment process is repeated (cf. Bain & Company, 
Rigby 2000). Thereby, the balanced scorecard ensures that the business is managed closely 
to its strategy and that the most crucial success factors are considered and not overlooked. 

The idea of the balanced scorecard has attracted a new stream of controlling and 
management literature (cf. Butler 1997; Friedag & Schmidt 1998a; Horvath 1999; Krahe 
1999; Fratschner 1999). However, little research has been done in translating the 
balanced scorecard to the field of evaluating executive education (cf. Pilz & Lee 2000). In 
terms of the development of our evaluation tool, STEER, several principles of the balanced 
scorecard have played an influencing role. 

Firstly, the new evaluation approach adopts the wider definition of value from the balanced 
scorecard. The balanced scorecard perceives value in a more holistic way. Thereby, value is 
created not only by financial processes, but also by many different, often intangible aspects 
of the organisation. As we have seen in the Theoretical Frame, several existing evaluation 
approaches focus solely on assessing the monetary value of executive education. STEER in 
contrast attempts to include wider aspects of executive education in its analysis, for example 
considering the networking among participants. In line with the balanced scorecard, STEER 
emphasises the interplay of financial and non-financial aspects of executive education. 

Secondly, STEER balances different perspectives within its measurement system in the 
same way that the balanced scorecard does. The underlying rationale of balancing the 
measurement system follows the broadened view on value. The balanced scorecard forces 
management to analyse different aspects of the business through its four perspectives. 
These perspectives are financial and non-financial, tangible and intangible, and past and 
future oriented. STEER takes on the idea of a balanced structure and translates it to the field 
of evaluation. Similar to the balanced scorecard, STEER structures its evaluation efforts to 
different evaluation perspectives. It distinguishes four perspectives, the stakeholder, the 
content, the process and the business perspective. Thereby, STEER achieves a 
comprehensive analysis of all the important aspects of evaluating executive education. 

Thirdly, the balanced scorecard has contributed the idea of relying on performance 
indicators to the new evaluation approach (cf. also ‘Bildungscontrolling’). The balanced 
scorecard advocates the use of simple performance indicators as the basis of its 
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measurement system. Each category therefore comprises a selection of key indices that 
reflect the current state of affairs. STEER relies on the same concept of indicator-based 
measurement. Thereby, it borrows the methodical framework of the balanced scorecard, but 
fills it with new, evaluation-relevant content. As a result, the quality of the measurement 
results depends to a large degree on the quality of the indicators used. In the case of 
STEER, these indicators have been chosen carefully with respect to management education, 
and have been defined clearly in terms of their scope (see our pilot testing, section 3.2.3). 

Fourthly, the integration of the measurement system to strategy is one of the key 
characteristics of the balanced scorecard. Although being designed for a different context, 
STEER aims to align its evaluation system to the strategic purpose of executive education 
as well. The balanced scorecard prescribes that the indicators of the four perspectives are 
adjusted to the vision and strategy of the organisation. The underlying rationale is that the 
indicators guide the decisions and activities of the organisation towards the strategic goals. 
Similarly, STEER considers the relevance of executive education with respect to strategy 
process capability in the framework of a corporate university (Deiser 1997). Additionally, 
STEER’s indicators and measures are selected with consideration of the goals of the human 
resource development strategy of the organisation. 

To conclude this subsection - the balanced scorecard has fathered the structure of STEER 
to a considerable degree. The strengths of the balanced scorecard - the balanced 
assessment, the use of simple indicators and the strategic alignment - have become 
important elements in the framework of our new evaluation approach. The purpose and the 
specific content of the balanced scorecard and STEER differ however. The balanced 
scorecard is a general management control tool, which assists in handling operations and 
aligning an organisation, whereas STEER focuses on evaluating executive education. 



3.2.2.2 Kirkpatrick’s Four Levels Evaluation Model 

“It [the Kirkpatrick’s model] provides a vocabulary and rough taxonomy for 
[evaluation] criteria. ” 



(Alliger & Janak 1989, p. 331) 
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The Four Level model of Kirkpatrick represents the second source in the development of 
STEER. Kirkpatrick’s ideas have shaped the new evaluation approach in a number of ways 
with respect to the content side of evaluation. 

The first contributing idea has been the recognition of the different levels of evaluation. 
Kirkpatrick’s model shows that the impact of training and development matures at different 
evaluation levels. Thereby, it is necessary to spread out the evaluation activity to different 
points in time. For example, reactions can be measured at the end of a programme, 
whereas behaviour requires some distance to the programme. In accordance with 
Kirkpatrick’s model, the methodology of the new tool embraces several evaluation 
sequences as well. Thereby, STEER attempts to capture the dynamics of the value of 
executive education over time and thus proposes measurement interventions before, during 
and after the executive programme. 

Secondly, the model of Kirkpatrick has assisted STEER in distinguishing between individual 
and organisational effects. Within Kirkpatrick’s framework, the Reaction Level can be 
regarded as having a predominantly individual emphasis, whereas the Results Level consists 
primarily of an analysis of the effects on the organisation. The Learning and Behaviour Level 
include a mixture of individual and organisational foci. As a result, Kirkpatrick’s model has 
influenced the new approach to take a systems view, i.e. STEER includes individual as well 
as organisational perspectives. Furthermore, STEER distinguishes between an internal and 
an external focus. 

Thirdly, the idea of Kirkpatrick’s Four Levels leads to the recognition that different evaluation 
methods might be necessary at the different levels. Kirkpatrick’s model advocates the use of 
multiple research methods for programme evaluation. STEER follows Kirkpatrick’s advice. 
The new approach uses quantitative as well as qualitative data. Furthermore, different 
techniques of data collection are devised. Kirkpatrick’s work points out the opportunity of 
using different sources for the evaluation information, which STEER considers as well. 
STEER thus combines four perspectives and covers different aspects of the value of 
executive education. 

Fourthly, the transfer of the learning to the job is emphasised. According to Kirkpatrick’s 
model, the transfer to the job is one of the keys to a programme’s value creation. The 
specific constellation of Learning - Behaviour - Results provides the framework to measure 
and explain the degree of this transfer (cf. also Baldwin & Ford 1988; Foxon 1994). The 
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transfer emphasis is taken on by the new evaluation tool. STEER evaluates to what extent 
transfer is possible and whether it actually materialises. 

In summary - STEER combines the balanced structure of the scorecard idea with 
Kirkpatrick’s taxonomy. The combination of the two is new in the field of evaluation research. 
One of the major reasons for that is presumably the relatively recent contribution of the 
balanced scorecard by Kaplan and Norton in the middle of the 1990s. 



3.2.3 Validating and Pilot Testing of STEER 

“ The design of the training evaluation determines the confidence that can be 
placed in its results. No training evaluation can be absolutely certain that the 
results of the evaluation are completely true. What the evaluator strives for is to 
use the most rigorous design possible (given the circumstances under which the 
evaluation occurs) to rule out alternative explanations for the results. ” 

(Noe 1998, p. 142) 

The last step in the development process has been the validation and pilot testing of STEER. 
In the following section, we briefly describe the validation procedures that we applied in the 
development process. Thereby, we distinguish the testing of content and the testing of 
methods. It is important to clarify the scope of our pilot testing. In the development process 
we have validated parts of STEER. Additionally, the first real-life application of STEER 
represents a second leg of its testing and is described in the empirical part in our thesis. 

Testing of Content 

STEER is structured into four perspectives of evaluation content. To validate the evaluation 
content (content validity), we tested the specific content in several stages beforehand. 

First of all, we interviewed the future users of the evaluation (cf. Patton 1997) in order to 
determine the purpose of the study. Given the formulation of the evaluation objective (cf. part 
3.1), we then consulted the literature (cf. chapter 2) and decided on the two principal 
conceptional sources. Consequently, we drafted a pool of evaluation measures and 
indicators, which represented the potential evaluation content of STEER. 
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Secondly, we discussed the pool of indicators with the users and other experts in the field. 
We captured the feedback of about ten users and experts (from various backgrounds). We 
used the responses to refine the indicator pool. 

Finally, we applied the indicators in a first test study to a programme in the DCU portfolio. We 
analysed the data collected and returned to the users a third time in order to assess the utility 
of our pilot results (cf. Patton 1997). Thereby, we captured first-hand practical feedback on 
our evaluation tool. The outcome of our content testing can be seen in the description of 
STEER in part 3.3. 

Testing of Methods 

As we can see in the next part, STEER comprises a selection of evaluation methods. In the 
development process we tested all of our methods and measures. 

In terms of primary research methods, e.g. interviews and questionnaires, we devised test 
studies. At first we discussed the structure and wording with people experienced in 
evaluation at the DCU (face validity). Subsequently, we posed our questions to test people 
from the actual population. Fortunately, there were two people in our business area, who had 
already participated in the programmes of interest. 45 In the testing process, we discussed the 
wording and structure and improved our questions. For example, complex and longer 
structures were shortened, difficult words were substituted by simpler and more common 
ones. The refinement procedures were also important for the translation of the questionnaire 
into German in order to match the tone and content to the English version. 

We conducted a similar testing with our questionnaires. Given the characteristics of our 
population 46 , we considered the piloting of methods as key to the later success of the tool. 
We were able to test our main follow-up questionnaire with the participants of one executive 
programme at DCU. Our pilot relied on email for sending out/back the questionnaire. The 
questionnaire itself was designed to be completed on-line and could be sent back by the click 
of a button at the end. The results of our pilot study were disappointing from a process view. 
Our response rate was only 21 percent (despite two reminders!). As a result we decided to 
conduct the questionnaire through the telephone, which resulted in a higher response rate 
(see subsection 4.2. 3. 2). 



45 In terms of our empirical study, we excluded the two test people from our population. 

46 We found executives notoriously short of time and extremely busy. 
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The results of the pilot-questionnaire assisted us in review the type of questions and their 
formulation. Therefore, we restructured the questionnaire and made the wording more 
understandable. Further, through the pilot testing we were able to confirm the reliability of 
our questionnaire. In terms of criterion and construct validity we were only able to use the 
experience of our testing procedures. The field, and partly the type of methods, disallowed us 
to use extensive statistical validation procedures. We tested the secondary research 
methods, e.g. analysis of documents and collection of administrative figures (see below) as 
well. Here, we determined whether the necessary sources were accessible by us and 
whether the generated data had a usable format. 

Overall, we tested all methods beforehand and used the insights to improve the evaluation 
approach. 



3.3 Description of STEER 

“Evaluation of training must be multi-level, customer focused, and support 
continuous improvement of training. ” 



(Brown 1997, p. 1) 

In the following part we present the conceptional contribution of our thesis. On the basis of 
the literature review, we can state that STEER represents a new approach. The balanced 
scorecard has neither been applied to the evaluation of executive education, nor has it been 
combined with the ideas of Kirkpatrick’s Four Level model. 



3.3.1 Underlying Conception 

The Strategic Tool for Evaluating Educational Returns (STEER) provides a systematic and 
differentiated evaluation methodology. It takes the form of four interlinked evaluation 
perspectives. 

The underlying rationale of STEER stresses the value of perspective in evaluation (cf. Hamel 
1996). Previous evaluation tools have often focused on one particular aspect of training and 
development. STEER covers all of the most important aspects of the value creation of 
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executive education through its perspectives. Thereby, STEER represents a kind of check- 
and-balance framework. Each of the four perspectives provides a separate assessment of 
the value of executive education. 




Business 

Perspective 



Stakeholder^ 

Perspective! 



Process 

Perspective 



Content 

Perspective: 



r THE 
STEERING 
t WHEEL i 



A presentation of important 
organisational measures and 
financial indicators to 
illustrate the wider 
impact of the 
p rag ra mine 



An evaluation of the 
way in which programmes 
are planned and conducted 
effectively and efficiently 



Measures of participant 
satisfaction and of the 
value derived for 
stakeholders 



An appraisal of 
programme content with 
respect to its strategic 
value, its timeliness and 
its rate of innovation 



Figure 3.2: The Strategic Tool for Evaluating Educational Returns (STEER) 



The combination of all four perspectives thus ensures a comprehensive assessment of the 
value of an executive programme. The evaluator can cross-check the results of each 
evaluation perspective with the other perspectives and receives therewith a more 
differentiated picture. For example, unexpected or extreme results in one perspective can 
be conditioned by the information of another perspective. Moreover, an impression given by 
one perspective can be consolidated by another. 



STEER is a strategic evaluation approach because it links executive education with 
corporate strategy. The specific goals of the executive programmes are derived from the 
overall SHRD goals (see the corporate university framework of Deiser (1997)) and can be 
found in the content and business perspectives. 



In general, we assume that the value generated by executive education can be captured 
through the stakeholder, content, processes and business perspective. In our view, the 
evaluation of executive education can best be compared to that of a puzzle. We 
acknowledge that it is impossible to measure the exact impact of executive education, but we 
discern that a well founded impression can be created by collecting different perspectives or 
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pieces of the evaluation puzzle. STEER covers the most important aspects and evaluates 
each perspective in turn in order to provide a systematic value assessment. The 
perspectives differ from each other in terms of their purpose and their point of view. 
Therefore, it enables the evaluator to keep track of the different perspectives over time. 



3.3.2 Four Evaluation Perspectives 

In the following subsections we introduce each perspective in detail. Thereby, we apply a 
four-step presentation procedure to all perspectives. First, the objectives assist us in 
understanding the purpose of the each perspective. Then, the terminology illustrates the 
specific focus, by defining and structuring the content of the perspective. Thirdly, evaluation 
instruments explain how the data is collected. Finally, expected results provide an overview 
of the kind of data generated by the perspective. 



3.3.2.1 The Stakeholder Perspective 



Rationale 

• Reflecting the value generated for the four 
principal stakeholders 

Measuring customer satisfaction 

• Focusing on the participants and their 
assessment of programme 

- Before, during and following the programme 


Terminology 

• Distinguishing four stakeholder groups 
Participants 

Superiors of participants 
Top management sponsors 
- Subordinates of participants 


Evaluation Instruments 

• Programme request forms 

• ‘Happy sheets’ (end-of-programme survey) 

• Follow-up telephone interviews 

• Face-to-face interviews with superiors, 
sponsors and subordinates 


Expected Results 

• Quantitative indicators 

Expectations and value assessments 
Benchmarks 

• Qualitative feedback 

- Perceived strengths and weaknesses 



Table 3.1 : STEER - The Stakeholder Perspective 
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3.3.2.1.1 Rationale 



The stakeholder perspective provides us with an important piece in the overall evaluation 
puzzle. Executive education is done in order to support the development of the management 
potential of a company. A corporate university thereby supplies a service to the organisation 
by conducting executive programmes. This service should satisfy the needs of the 
'customers’ of a corporate university (see below for the definition of customers) and thereby 
gain acceptance and credibility among its stakeholders. The underlying logic of the 
stakeholder perspective becomes clear, when we consider the negative example of customer 
dissatisfaction. If a corporate university programme does not fulfil the expectations of its 
customers, then the programme is likely to be considered ‘not worth going to’. As a result, 
potential executives will not invest their precious time in the programme - or, when obliged to 
take part, they will not give it their full attention . 47 Executive education can only achieve a 
strong impact if accepted and supported by its participants as well as by the other 
stakeholders. 

Therefore, the objective of the stakeholder perspective is to reflect the value created for the 
various stakeholder groups. It provides measures on the expectations, the needs and the 
satisfaction of the 'customers’ and other stakeholders of executive education. The 
perspective considers therewith the individual effects of executive education from an external 
view with respect to a corporate university . 48 

The stakeholder perspective combines quantitative and qualitative data. It takes a 
longitudinal approach, by collecting data before, during and after the programme. The main 
focus of the stakeholder perspective is on the participating executives, but it also considers 
the wider group of stakeholders. 

3.3.2.1.2 Terminology 

The Strategic Tool for Evaluating Educational Returns assumes a broad view of customers. 
It considers the participants as direct customers, but also includes other stakeholders as 
relevant indirect customers. On the whole, the tool discerns four stakeholder groups. 



47 The lack of attention can be seen for example by participants leaving early from programmes or being engaged 
in constant telephone calls with their office at home. 

Here, ‘external’ implies that most of the data is sourced from outside the corporate university, i.e. from 
stakeholders like the participants or the sponsors of the programme (who are nevertheless within the 
organisation). 
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STEER focuses on the participating executives from the middle management. 
Management programmes specifically aim at the development of the skills, knowledge and 
mentality (cf. 2.2. 1.2) of the participating executives, who are the traditional key customers. 
Most of the potential value from the investment in executive education is expected to be 
created by or through participants when they return to their jobs. 

Another important stakeholder group is the top management (cf. subsection 2.2. 1.1), who 
frequently sponsors the executive education. Management programmes can be funded by 
high level sponsors on the basis of specifically defined strategic objectives. Considering the 
underlying objective of building executive capabilities in the organisation, the top 
management has a clear stake in the success of the executive education as well. 

Furthermore, we consider the involved superiors of participants as important stakeholders. 
Superiors hold a considerable stake in executive education since they often initiate the 
education of their managers. 49 For them, sending subordinates to a programme implies a 
considerable investment in terms of opportunity costs, with the executive being away from 
his/her work. 

Finally, the subordinates of participants must be taken into account. Just as the group of 
involved superiors, subordinates have similar stakes in the executive education. In their daily 
work the capabilities of ‘their’ executive is likely to affect their motivation and performance. 
Therefore, a successful programme that improves the skills, knowledge and mentality of the 
executive can be of great value to them as well. 

Overall, the extent to which all four groups can be included in the evaluation depends on the 
specific purpose of the assessment and in particular on the available resources in the 
organisation. 

3.3.2.1.3 Evaluation Instruments 

In the framework of STEER we recommend that different evaluation instruments be used 
with respect to the various stakeholder groups. 

The participating executives should be approached three times in the evaluation - before, 
during, and several months after the programme. We suggest after several months, i.e. from 



49 In some cases, the superiors belong to top management, i.e. the participants themselves are high ranking 
managers already and thus their superior is a board member. 
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six months to two years, because the participants should have the time to transfer their 
learning to the workplace. 

The start of the evaluation cycle is marked by a request form, which participants use 
together with their supervisors to specify their needs in terms of capabilities. The request 
form can be a simple pen-and-paper questionnaire, a more sophisticated online survey or 
even a short, standardised telephone interview. The request form captures the needs and 
expectations of participants. It presents, thereby, an important reference point for comparing 
subsequent changes in skills, knowledge and mentality after the programme. 

During the last part of a programme participants are given so called ‘happy sheets’ (cf. 
Fischer & Weinberg 1988), which ensure that the participants’ Reactions are collected (cf. 
Kirkpatrick 1979). These end-of-programme surveys usually take the form of a short pen- 
and-paper questionnaire filled in by participants. A short feedback session can provide 
additional qualitative data on the programme. Many existing executive education 
programmes include end-of-programme questionnaires at some point (cf. Hackett 1997). 




Figure 3.3: STEER - The Different Interim Points of Approaching Participants 



The most important participants’ survey is done several months after the programme. It is 
called follow-up survey to mark the end of a executive education sequence. The focus of 
this questionnaire is on the changes in skills, knowledge and mentality that have occurred 
back at the workplace due to the programme. Primarily, the transfer of learning is addressed. 
In addition, the fulfilment of participants’ needs and expectations - as specified in the request 
form - can be assessed. The follow-up survey can be a mail questionnaire or a standardised 
interview. 

In addition, to the three questionnaires, data from the organisation’s performance appraisal 
system can be collected. Ideally, the system would provide an assessment of the 
participants’ capabilities before and after the participation in the programme. This secondary 
data could then complement the primary data from the participants themselves. 
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We consider semi-structured interviews as the most appropriate form to collect primary data 
from all the remaining stakeholders - the sponsors, superiors and subordinates. 50 Ideally, 
interviews should be spread before and after the programme, in order to be able to identify 
change that has occurred. 

Sponsors are interviewed in order to gather their responses on the realisation of the 
management programmes. Frequently, sponsors themselves provide active input to a 
programme, i.e. through a speech or presentation. Therefore, an informal interview 
provides the opportunity to check, to what extent the programme created value in the eyes of 
its sponsors. 

The purpose of the superior and subordinate interviews is to collect qualitative data on the 
behavioural changes of participating executives, i.e. 360 degree feedback. The assessment 
by those two groups complements the picture on the programme impact. Furthermore, 
superior and subordinate interviews can validate the responses given by participants in the 
follow-up interviews. 

In conclusion, the stakeholder perspective includes interviews and questionnaires with 
middle managers and executives of the organisation. Given the maxim of feasibility, simple 
and easy-to-administer questionnaires and interviews should be chosen. 

3.3.2.1.4 Expected Results 

In terms of evaluation measures, we suggest a collection of indicators per perspective (from 
which a selection can be made in practice). We consider a selection of four to five measures 
per perspective best for the later implementation. Given four perspectives, a total of 16 to 20 
indicators remain manageable in the evaluation. A higher number of indicators complicates 
the evaluation and makes the measurement process more cumbersome. 

Given the measurement instruments above, the stakeholder perspective should generate a 
mixture of quantitative as well as qualitative results. The results of the stakeholder 
perspective could thus contain information on the following dimensions. 



50 Interviewing subordinates should be planned and conducted carefully. Most German companies have strict 
union laws, which require a codetermination by union officials when interviewing subordinates. Most executives of 
the middle management are not unionised and therefore do not require co-ordination with a union official. 



122 




Quantitative Indicators 

These could take the form of ratings, totals or percentages. 

• Overall value rating of participants 

• Benchmark to other programmes as seen by participants 

• Call-back 51 and cancellation rates by participants 

• Attendance rates of sponsors at programmes 
Qualitative Measures 

• Fulfilment of participants’ expectations 

• Degree of satisfaction of sponsors, superiors and subordinates 

• Changes in performance ratings as assessed by an internal performance appraisal 
system 

Table 3.2: STEER - Measures of the Stakeholder Perspective 

Overall, the stakeholder perspective assumes that value is created by meeting the needs of 
the different stakeholders of executive education. With its quantitative and qualitative 
measures, the stakeholder perspective provides an overview of the assessment from all 
stakeholder groups. Thereby, the evaluator can judge the perceived benefits of executive 
education from the stakeholder perspective. 



3.3.2.2 The Content Perspective 



Rationale 

• Assessing the content value of the 
programme with respect to the following 
aspects 

Strategic relevance to the organisation 
Applicability and transfer for participants 
Customisation in terms of HRD needs 
Innovation and further development 



Terminology 

• Assessing the subject matter as well as the 
teaching methods 

• Focusing on relevance, applicability, 
customisation and degree of innovation with 
respect to the three elements of capability 

- Skills 

Knowledge 

Mentality 



(continued on the next page) 



51 The “call-back rate” implies that executives take part in more than one programme in the portfolio of the 
corporate university. “They return for more" so to speak. 
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Evaluation Instruments 


Expected Results 


• Follow-up telephone interviews 


• Quantitative indicators 


• Standardised summaries per programme 


Degree of customisation/topic frequency 


• Face-to-face interviews with programme 


Perceived relevance & applicability 
Assessed learning 


organisers 


• Qualitative feedback 


• Talks with external experts 


- Assessment of content with respect to skills, 
knowledge and mentality 



Table 3.3: STEER - The Content Perspective 



3.3.2.2.1 Rationale 

The content perspective contributes an assessment of the input side of executive 
programmes. It appraises the strategic value of the content of executive education with 
respect to its relevance, applicability, degree of customisation and rate of innovation (for a 
definition see below). The underlying rationale is that an executive programme creates value 
by the ‘right’ content and method. In other words, the selection of topics and their focus is 
one of the key success factors of executive education. 

We acknowledge that the evaluation of content can be demand-driven (“pull”) and supply- 
driven (“push” - cf. Paauwe & Williams 2001). In the former case, participants assess the 
value of content with respect to their specific needs. Thereby, they demand the right 
programme content based on their view of the relevant topics. For the evaluation, this implies 
that the participants provide feedback on the content of the programme. In the latter case, 
the supply side of executive education, i.e. the educators, evaluate the content of the 
executive programme. Thereby, the supply side might have a different opinion on the 
importance of certain topics. Educators possess the professional knowledge about the value 
of teaching content. 

Overall, the content perspective assesses the value with respect to the programme content 
elements and, thereby, takes foremost an internal perspective. 52 It ensures that not only 
popular topics are taught, but also those topics, which are important in the wider framework 
of corporate strategy. 



52 ‘Internal’ applies to the fact, that an important share of the data is collected internally with respect to the 
corporate university. 
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3. 3. 2. 2. 2 Terminology 



In the framework of STEER, the term content possesses a specific connotation. Therefore, 
we provide a short definition of content in order to explain the term and to render the 
perspective more understandable. 

First of all, content comprises both the subject matter of executive education, ‘what’ is 
taught, and the methods used in the programmes, i.e. ‘how’ the subject matter is taught. The 
subject matter covers the topics and themes of a programme, as for example Value Based 
Management. Ideally, the subject matter should be aligned to desired skills, knowledge and 
mentality. The content perspective takes a look at the underlying methods as well, i.e. the 
learning architecture. For example, potential methods could be an interactive lecture or a 
practical simulation. The Theoretical Frame has illustrated the way capabilities can be taught 
(cf. Schwuchow 1992; Garavan et al. 1999b). 

Secondly, the focus is on the relevance, the applicability, the degree of customisation 
and the rate of innovation of content. We consider these four characteristics as crucial for 
the development of an executive’s skills, knowledge and mentality for the following reasons. 

The relevance of content reflects the importance of a topic with respect to participants’ 
responsibilities. Thereby, we distinguish overall relevance, determined by the characteristics 
of the company and its environment, from specific relevance that applies directly to the job 
of a participant. Applicability refers to the degree to which the learning is transferable to the 
participants’ responsibilities. The degree of customisation indicates the extent to which a 
programme’s content has been aligned to suit the specific needs of the participants and, in a 
wider sense, the needs of the company. Customisation is a critical issue for a corporate 
university when external providers are used for the delivery of programmes. Finally, the rate 
of innovation is an important feature of executive education. The life-cycle of our knowledge 
today is shortening, therefore management programmes can provide value through the 
constant innovation of content and methods. 

Overall, the content side takes a look at the subject matter and the teaching methods of 
executive education with respect to specifically defined quality features. 

3. 3. 2. 2.3 Evaluation Instruments 

The content perspective proposes a different composition of measurement instruments 
compared to the stakeholder perspective. The collection methods for the content side can be 
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broadly distinguished by its sources. On the one hand, there is a variety of data that exists 
as part of the daily operations of executive education, i.e. internal sources. On the other 
hand, some information must be gathered from outside the department or unit, i.e. external 
sources. 

The internal sources rely on the input of programme organisers and teachers in order to 
provide some of the content measures. Organisers and teachers are most qualified in terms 
of executive education. Their judgement on the effects of a programme relies on a profound 
knowledge and experience in the field. Yet, these internal sources of evaluation data should 
be considered carefully. The judgement of staff can easily be biased due to their 
involvement in executive education. We therefore attempt to balance the internal input by 
complementing it with other external perspectives. Consequently, the tool does not solely 
rely on the responses of organisers and teachers of executive education. 

The collection of internal data consists of short, standardised summaries of each 
programme that are completed by organisers and teachers. These summaries capture the 
most important content statistics of the respective executive programme and include key 
comments 53 , i.e. they contain topics covered and methods used. Additionally, they list 
information on speakers and the structure of the programme agenda (i.e. duration and 
intensity). Thereby, a pool of content indicators can be compiled and analysed. Ideally, these 
summaries are supported by information technology, so that a later analysis can be done in a 
time and resource efficient way. 

A second stream of data is referenced from external sources. This implies that the content 
is evaluated by people who are not working at or for a corporate university. We discern two 
external groups that can provide further assessment. 

The first group comprises the different stakeholders mentioned above in the stakeholder 
perspective. The collection from this group would not require additional evaluation efforts as 
it is collected through existing instruments (see also subsection 3.4. 1.1). For example, the 
instruments described in the stakeholder perspective, i.e. surveys and interviews, could 
comprise elements that evaluate the content of a programme as well. Participants thereby 
provide a judgement on the relevance of different topics and methods and sponsors assess 
the strategic relevance of overarching programme themes. 



53 In order to make this work - these summaries should not be hours of additional work for the respective 
organisers and teachers. It should take the form of a brief, standardised programme diary. 
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External experts represent the second group of independent content evaluators. The 
principal idea of this group is to provide an outside-the-company view on the content of 
executive education. The group must not be large. It could consist just of a lose network of 
selected academics, consultants and other experts from various backgrounds. It is Important, 
however, that confidentiality is guaranteed with all members of this group - in particular, due 
to the fact that the content most likely includes strategically sensitive data. The feedback 
from this group would be of a qualitative nature, collected through a scheduled meeting or a 
series of interviews. The idea is to ask external experts for their opinion on the content ca. 
once a year. 

Overall, internal sources are meant to provide the qualitative data on content statistics 
complemented with some comments of people with an external perspective. 

3.3.2.2.4 Expected Results 

The content perspective includes measures on the subject matter and the method of the 
executive education. This data is of a quantitative nature. Secondly, the perspective consists 
of qualitative feedback from related stakeholders and outside-the-company experts. 

Content Statistics (Quantitative) 

• Relevance and applicability ratings by participants 

• Analysis of the degree of customisation of content (using a rating-code, cf. subsection 
5 . 1 . 2 . 2 . 1 ) 

• Innovation assessment - rate of new topics per year 

Content Feedback (Qualitative) 

• Feedback from participants on their personal and professional learning with respect to 
skills, knowledge and mentality 

• Strategic relevance assessed by sponsors 

• Assessment of subject matter and methods by external experts (external benchmark) 

• Internal feedback from programme organisers and teachers 

Table 3.4: STEER - Measures of the Content Perspective 

Overall, the content perspective assumes that value is generated by relevant input, 
delivered with the appropriate method at the right time. Therefore, the perspective provides 
the evaluator with insights on the value of the subject matter and method of an executive 
programme. 
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3. 3. 2. 3 The Process Perspective 



Rationale 

• Evaluating the effectiveness of the 
programme with respect to the overall 
design and structure 

• Reviewing the efficiency of programme 
organisation and delivery 


Terminology 

• Discerning three important parts in the 
process of a programme 
Development 
Organisation 
Delivery 


Evaluation Instruments 


Expected Results 


• Analysis of operational performance 


• Quantitative indicators 


statistics 


Programmes’ and participants’ statistics 


- Annual administrative data of internal 


Mix and attendance rates of speakers 


support staff 


Programme timing and schedules 


- Programme specific data 


• Qualitative feedback 


• Evaluation of programme cycle times 


- Perceived strengths and weaknesses 


• Follow-up telephone interviews 


- Suggestions for improvement 



Table 3.5: STEER - The Process Perspective 

3.3.2.3.1 Rationale 



The process perspective investigates whether executive education is planned and conducted 
effectively and efficiently. It attempts to provide an assessment of the operational 
excellence (Becker & Huselid 1999) of the delivery of executive education. Thereby, we 
assume that value for stakeholders can be created by process effectiveness and efficiency. 
The underlying rationale can be seen in a quote by Brinkerhoff (1987, p. 34) - " There should 
be solid evidence that a programme is efficient, that it does not use more resources than 
necessary, that it is carried out quickly with the least disruption possible, that it achieves 
learning gains in proper and enjoyable ways." In addition to the right content, we can see that 
the success of corporate education depends also on the right way that the programme is 
planned and delivered. Therefore, STEER evaluates the processes of planning and 
delivering executive education from an internal point of view. 

3.3.2.3.2 Terminology 

Process is a generic term for several activities, which prepare, support and realise executive 
programmes. In order to clarify the term, we explain the most important components below. 
These are the development, organisation and delivery processes of a programme. 
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The development of new programmes, or new elements of existing ones, represents an 
important part of the process of executive education. Hence, the process perspective 
considers the kind of resources used for the design and planning of programmes. We 
assume that a faster and more focused process is of value. Shorter programme 
development cycles free resources and enable a greater responsiveness to new topics. For 
example, if the creation of a new executive programme takes several months, then the 
programme content cannot respond to current corporate issues due to its development lag. 

Secondly, process comprises the organisation of executive education, as for example event 
management. Organisation refers to all non-content support of executive education. 
Participants must be selected, contacted and informed about their programmes. Especially 
with respect to the full schedules of executives, these activities are not to be underestimated. 
For example, the quality of a programme is likely to suffer if participants arrive unprepared or 
if - due to cancellations - only half of the participant group takes part. Programmes often 
have an optimal size, which should be achieved. We assume that the right programme 
size and the right mix of backgrounds proves beneficial to an executive programme. For 
example, if a programme is designed for Vice Presidents, then the participant group should 
primarily consist of Vice Presidents in order to enable the sharing of experiences and 
networking. Moreover, the content is then targeted for Vice Presidents. We assume that 
participants from other management layers would not derive the same value from the 
programme than those members of the target group. We assume further that the corporate 
value of a programme depends directly on how many executives were exposed to it. From an 
organisational perspective, the participants are the multipliers of the learning from the 
programme . 54 

Finally, the process perspective contains the delivery of executive education. Delivery 
focuses on the effective and efficient realisation of the programme itself. It assesses the way 
in which the learning (and other) goals of a programme are achieved. For example, it takes a 
look at the performance of teaching staff and at the actual programme flow. Hereby, we 
assume that a disciplined and well-run programme delivery provides value to its customers. 

Combined, the process perspective focuses on the value chain of the executive education 
process. The process perspective assesses the service of developing, organising and 
delivering executive education. 

54 Clearly, we cannot assume a linear relationship of ‘the more participants the better J . Yet a corporate university 
must make sure that they reach their optimal programme group sizes. Empty spaces in executive programmes 
are a waste of direct costs and can entail lost opportunities as well, as for example, in respect to networking. 
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3. 3. 2. 3. 3 Evaluation Instruments 



Spitzer (1999) points out that an imbalance exists for most evaluation instruments. There is 
an overemphasis on work sheets, questionnaires and similar tools, but the wealth of 
operational performance measures is often ignored. This process perspective evaluates the 
internal activities of a corporate university. Thereby, the perspective applies primarily 
quantitative controlling measurement techniques, which can be distinguished into methods 
that are gathered centrally and de-centrally. 

In order to illustrate the value creation, the process perspective proposes that some 
measures are gathered centrally with respect to the executive education activities. It is 
likely that these measures exist in some form or another in any corporate university - most 
likely in the area of controlling or budgeting. Therefore, the appropriate sources must be 
identified and the necessary data compiled. The centrally gathered measures consist of 
statistics that provide information on the resources used for a programme portfolio of a 
corporate university. These could include data on the development activities and the 
organisation. Ideally, IT support would be available to extract these measures. Those 
indicators (see expected results below) that are not yet part of any controlling or IT system, 
must therefore be gathered by hand. Generally, the process perspective suggests that 
standards are defined for each programme type, which outline the procedure of gathering the 
appropriate data. Once in place, the controlling process could then take the form of 
standardised, periodical measurements or end-of-year reviews. 

The second part of process data is collected de-centrally by the personnel working in 
organisation and in delivery. Here, the process perspective can share measurement 
instruments of other perspectives. For example short programme summaries of the content 
perspective can provide data for the process perspective as well. Members of the event 
management/organisation can compile a brief summary sheet for each executive programme 
realised - capturing a selection of statistics, as for example capacities used. These can later 
be aggregated and analysed across programmes. 55 

Further process measures can be collected through stakeholder feedback. Assessment 
data on lengths and intensity by participants are relevant process measures, which complete 



55 Ideally, a flexible Information Technology system could leverage a lot of work. For example, software used for 
event organisation could comprise an evaluation module as well, then many of the required measures could be 
analysed at the click of a mouse button. Consequently all descriptive data on programmes need no longer be 
collected separately, but can be summarised automatically. 
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the process perspective. A selection of questions can be included in the follow-up 
questionnaires for participants. 

Overall, we suggest that the process data is gathered through conventional controlling 
techniques. Part of the data is derived centrally, most of the process measures, however, are 
gathered de-centrally. 

3. 3. 2. 3.4 Expected Results 

The process perspective provides measures of the effectiveness and efficiency of executive 
education. The results of this perspective are primarily of a quantitative nature and can 
include the following measures. 

Centrally Gathered Data 

• Totals and averages of programmes realised - classified according to programme types 
and frequencies 

• Percentages of target groups reached by each programme 

• Productivity ratio of participant days per member of staff 

• Duration and timing assessment of the programme by participants 

• Development cycle times of new programmes or of new strategic topics 
De-centrally Gathered Data 

• Programme capacities used/not used 

• Strengths, weaknesses and improvements from feedback of participants 

• Totals and averages on participants contacted/trained/qualified according to programme/ 
level/function 

Table 3.6: STEER - Measures of the Process Perspective 

Overall, the process perspective takes a look behind the scenes and evaluates the 
development, the organisation and the delivery of executive programmes. Thereby, the 
perspective complements the overall evaluation puzzle by focusing on the effectiveness and 
efficiency of programmes. 
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3. 3.2.4 The Business Perspective 



Rationale 


Terminology 


• Assessing the value contribution of the 


• Distinguishing between 


programme to the organisation 


Organisational outcomes 


Reviewing organisational effects 


Accountable costs 


Selecting financial indicators 


Financial benefits 


Drafting a differentiated cost structure 


• Using data which is 


• Taking a broader organisational view on the 


Measurement based 


effects of the programme 


Estimation based 


Evaluation Instruments 


Expected Results 


• Cost accounting and controlling analysis 


• Quantitative indicators 


• Operational statistics of the training 
department 

• Data on performance appraisals 


Overview of the cost structure 
Degree of networking 
Retention rate of executives 
Assessment of the monetary value of the 


• Follow-up telephone interviews 


programme 




• Qualitative feedback 




Perceived contribution of programme 



Table 3.7: STEER - The Business Perspective 

3. 3. 2.4.1 Rationale 

The underlying rationale of the business perspective is that executive education is not only 
done for individual purposes, but also pursues important organisational goals. 
Organisations invest in management development in order to succeed in their businesses. 

The three preceding perspectives - stakeholder, content and process - attempt to evaluate 
how far valuable skills, knowledge and mentality have been developed by the executive 
programme. The logic of these three perspectives is that well developed and relevant 
capabilities lead to a competitive advantage, which is of value to the organisation. The 
business perspective takes a broader look at the impact of executive programmes. In other 
words, it evaluates the wider effects that are achieved apart from building capabilities. We 
thus try to capture the value of executive education on the organisational level and take, 
thereby, an external perspective. The business perspective combines the presentation of 
organisational and financial measures that indicate the value contribution of executive 
education. 
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3. 3. 2.4.2 Terminology 



The business perspective represents a selection of ideas and terms. It follows closely 
Kirkpatrick’s (1979) notion of the fourth level Results. We explain the key terms of the 
perspective in the following in order to enable a better comprehension of the different 
elements. In the framework of the business perspective, we distinguish accountable costs 
and financial benefits. However, as seen in the Theoretical Part, the exact measurement of 
financial data is difficult. Therefore, the business perspective contains another, broader 
element - organisational outcomes. 

In the framework of STEER we define accountable costs as all fixed and variable expenses 
related to the development, planning and delivery of executive education. 

Section 2.3.4 has illustrated the difficulty of arriving at a clear cost understanding of 
executive education. Van Adelsberg’s pyramid (1999) demonstrated the iceberg-nature of 
costs. The direct and visible expenses can be collected and summed up, whereas estimating 
the lost opportunity costs seems difficult because it does not rely on any concrete figures. 
We base our definition of accountable costs on the cost understanding of Van Adelsberg. We 
assume that accountable costs comprise all “Visible Costs". Furthermore, we simplify Van 
Adelsberg’s “ Hidden Costs" by the following assumption. We presume that on average an 
organisation pays an executive the amount as wages, which is considered equivalent to 
his/her value contribution or productivity (cf. Stolovitch & Maurice 1998). 56 For example, an 
executive earns EURO 250,000 per annum and therefore we assume that his/her 
contribution to the firm during one year is at least the equivalent of this sum. Consequently, 
the hidden opportunity costs are assumed to equal the salary of the executive for the time of 
the programme. If our particular executive would take part in a two-week programme, the 
“Hidden Cost” would accumulate to EURO 9,600. 57 

Overall, accountable costs can therefore be distinguished into two parts. First of all, there is 
the visible expense part, which can be calculated by the actual fixed and variable costs. And 
secondly, there is a hidden expense part consisting of a standardised estimation of the lost 
productivity being away from the job. 



56 We acknowledge that there exists a wealth of theories in the field of economics and management that would 
disregard this assumption on the basis of market inefficiencies, stickiness of demand and supply, union influences 
and life-long incentives. 

57 EURO 250,000 is the annual salary of the executive. A year consists of 52 weeks and therefore the executive 
earns roughly EURO 4,800 per week. The opportunity loss for two weeks thus is EURO 9,600. 
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The financial benefits of training and development represent the most debated area of 
evaluation. In section 2.3.4 we described the various problems in measuring the benefits 
objectively. No accepted measurement standard exists for executive education. Most work 
relies on perceptions of participants or involved staff. In the context of STEER, we define 
benefits as those outcomes, which have materialised as a result of the executive 
programme. Likewise, following the inverse logic, benefits are those outcomes, which would 
not have materialised without the intervention. This implies that as soon as there is evidence 
that executive education contributed to an outcome, it can be counted as a benefit of it. For 
example, if two executives meet during an intervention for the first time and decide to initiate 
a new project, then the outcome of the project would be understood as the benefit of that 
programme. Although the intervention might not cause all the financial benefits of the project 
directly, without the intervention, there would be no project at all. 

Therefore, the difference to other evaluation approaches is, that STEER does not rely on 
experimental proof for the analysis of benefits. We assume that the entire benefit, financial 
and non-financial, can never be determined exactly. Using the analogy of a puzzle from the 
beginning, gathering a sufficient number of pieces can be enough to identify the overall 
picture. In evaluating executive education it is very unlikely that all puzzle pieces are found 
and fitted together, but with some effort an overall impression emerges. 

Finally the business perspective considers organisational outcomes. Organisational 
outcomes comprise broader effects on corporate culture, turnover, morale and motivation. 
These outcomes face the same cause-and-effect difficulty as the financial benefits. 

Strikingly, very few executive programmes possess clearly formulated financial goals. Most 
interventions have identified behavioural or cognitive aims, as for example developing a 
certain skill or awareness. Therefore, the business perspective perceives an analysis, based 
only on the costs and benefits, as insufficient. Subsequently, organisational outcomes aim at 
capturing the more organisational, intangible effects of executive education. These are 
benefits, which cannot be measured in monetary terms. We include organisational outcomes 
in order to complete the overall picture of the evaluation and to understand underlying 
effects. 

In total, the business perspective covers the costs as well as the benefits. Costs might be 
visible or hidden. Benefits might be financial (narrow understanding) or organisational (broad 
understanding). 
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3. 3. 2.4.3 Evaluation Instruments 



The business perspective relies on a set of measurement techniques, which can be broadly 
distinguished into measurement-based and estimation-based. The difference lies in the 
source of the information. Measurement-based data can be observed and collected by the 
evaluating entity directly. Estimation-based data requires interviews (or surveys) of 
intermediary people who provide their estimations of the outcome. 

In terms of the measurement-based instruments, cost accounting plays an important role. 
Conventional accounting and controlling procedures can capture the accountable costs of 
executive education. All expenses are usually recorded during the year and can therefore be 
referenced with the help of the controlling personnel. For example, the fixed costs of a 
corporate university can serve as the basis to calculate the overhead, which is then allocated 
to different programmes. The charges of external partners for the development and delivery 
as well as accommodation expenses for example would represent the variable costs, which 
can be allocated to the respective programmes. The aim is to attain an overall cost structure 
in order to understand the dimensions of programme costs and to compare different types of 
executive education with each other. 

Few instruments exist for measuring the financial benefits of executive education in 
general. In the case of technical training, an evaluation can measure the monetary impact of 
the education. For example, a training to improve driving skills of the mobile service staff can 
reduce the number of accidents and thereby incurred costs. A training on the operation of a 
machine can result in higher output and less scrap. In contrast, abstract and cognitive 
executive programmes provide almost no opportunity to measure their financial impact - 
although they might be estimated (see below). 

Nevertheless, organisational outcomes can be measured. Changes in the behaviour of 
executives can be either observed or derived from existing performance appraisal data within 
the organisation. For example, the participant retention rate can be determined, i.e. whether 
the executive programme has an effect on the rate of executives leaving the organisation. 
The underlying logic here would be that an executive programme can serve as reward and 
motivation. Furthermore, the degree of networking achieved by the participants of the 
programme is another example of an organisational outcome. 

We include several estimation-based instruments to illustrate the value creation through 
executive programmes. These substitute for the lack of measurement-based instruments in 
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the case of the financial benefit. The underlying logic is that, although many financial 
benefits cannot be measured, they can be estimated objectively (Gulpen 1996). 

The estimation of the financial benefit implies that participants and other stakeholders assess 
the value of executive education in the framework of telephone or face-to-face interviews. 
For example, participants can be asked to state the amount that they would be willing to 
invest in the programme, if they had to. Thereby, an estimated market value of a programme 
can be established, that could provide some indication on the worth of an executive 
programme. The underlying logic for using estimations by participants is that executives 
frequently have to make financial judgement in their daily work. Therefore, we assume that 
the participating executives possess quite a good sense for investment decisions. 58 

We are confident that well-founded estimations can be of use to the evaluation of executive 
programmes. When compiled carefully and conservatively, estimations can provide 
interesting data in the framework of evaluation. Moreover, we consider estimates as 
additional pieces in the evaluation puzzle. Estimated values are not looked at in isolation, but 
are put into perspective in the framework of STEER. 

To conclude, the business perspective relies on measured and estimated data. The 
measurement side is likely to come from internal data sources, which exist independently 
from the evaluation. Estimations are done in the framework of follow-up interviews and 
questionnaires. 

3.3.2.4.4 Expected Results 

The business perspective provides data according to the three categories described above - 
accountable costs, financial benefits and organisational outcomes. 

Accountable Costs 

• Direct (and indirect) costs of executive education 

• Benchmark of gained/lost savings with respect to external programmes 
Financial Benefits 

• Value of the programme as assessed by participants 

58 The measurement of perceived financial outcomes has been frequently criticised as inexact and unreliable. 
Interestingly, other organisational functions have fewer problems justifying their estimations. A lot of the 
accounting procedures are based on standardised conventions which consist of estimations as well. For example, 
the assessment of goodwill or the valuation of brands rely predominantly on estimations. The process of due 
diligence involves estimations in order to determine the value of a company. 
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• Effectiveness and efficiency gains as assessed by participants 

• Results from the transfer to the job and from implemented business projects, estimated 
by participants 

Organisational Outcomes 

• Retention rate of former participants remaining in the organisation (compared to general 
turnover of executives) 

• Degree of networking between participants 

• Observed changes in skills, knowledge and mentality by superiors and subordinates 

• Fulfilment of Human Resource Development goals 

Table 3.8: STEER - Measures of the Business Perspective 

Overall, the business perspective aims at evaluating the impact from a non-educational point 
of view. The assumption is that executive education creates direct value for the entire 
business on an organisational level. 



3.4 Implementation of STEER 

“In the minds of many practitioners evaluation is viewed as a problem rather than 
a solution, and an end rather than a means. ” 



(Foxon 1989, p. 2) 

In part 3.3 we have illustrated the underlying ideas of STEER. We have explained the 
purpose and content of the four different perspectives. The present part outlines the 
underlying ideas of the implementation of STEER. Thereby, two important steps are 
distinguished. In section 3.4.1 we depict the implementation of the tool in terms of the 
procedures within a corporate university. In section 3.4.2 we consider important 
organisational interfaces that must be established and maintained during the evaluation. 

We would like to point out that the following description serves as part of the general 
explanation of the Strategic Tool for Evaluating Educational Returns. Later, in the empirical 
part of this thesis we apply the new approach to the executive programmes at the 
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DaimlerChrysler Corporate University. We outline the actual application of STEER in 
much more detail in part 4.2. 



3.4.1 Methodical Framework and Procedures 

In the framework of STEER, we distinguish between data collection and data analysis. The 
following ideas are meant to assist in understanding the new evaluation approach. 

3.4.1. 1 Data Collection 

As we have seen above, several different sources of data exist in the framework of STEER. 
The underlying idea of STEER is to use a set of evaluation instruments that serve all of 
the four perspectives. For example, the telephone interviews provide data to all four 
perspectives. At the same time, face-to-face interviews deliver data to three of the four 
perspectives. 

The reduction of the number of instruments avoids the duplication of effort of having a 
separate set of instruments per perspective. Moreover, it reduces the complexity of 
suggested instruments and renders STEER more feasible on the whole. 




Figure 3.4: STEER - The Underlying Idea of the Use of Evaluation Instruments 

Generally, the new evaluation approach distinguishes between three ways of using the data 
collection instruments - continuous , programme specific and annual. 

On the one hand, STEER contains a set of tools that are applied constantly throughout the 
evaluation cycle (which is usually defined as the calendar year). These include the 
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continuous input from programme organisers and teaching staff as well . 59 Secondly, there 
are the interviews and questionnaires, which follow the programme cycle. This implies that 
‘happy sheets’ are filled out at the end of each programme. Follow-up interviews with 
participants and superiors are done several months after the executive programme. Finally, 
there are annual activities of data collection. These include interviews with superiors and 
sponsors and the annual performance appraisal procedures . 60 For example, during the 
annual feedback talk between an executive and his/her supervisor, the effectiveness of 
executive education is assessed with respect to the achieved goals and performance. 
Additionally, the review of programmes’ content by external experts can be done at the end 
of the year. Furthermore, all kinds of internal and external financial analysis is done annually. 
For example, the yearly cost accounting procedure can feed important financial background 
data into the business perspective. 

Most of the collection activities must be initiated and co-ordinated through a central 
evaluation entity in order to ensure timely data flow. Therefore, it seems meaningful to assign 
one person to take on the overall responsibility of running the evaluation efforts. 



3.4.1. 2 Data Analysis 

In our framework, evaluation data is collected by several instruments and at different times. 
The various data streams resulting from the various data collection techniques are reviewed 
at the end of the evaluation cycle (i.e. after the year has finished). Ideally, all data streams 
take the form of compatible electronic files, which can be joined in the evaluation, e.g. in an 
Excel sheet. The batch of raw data then needs to be analysed according to the four 
perspectives. Thereby, one can distinguish between quantitative data and qualitative 
feedback. 

The quantitative data is prepared according to a set of principles, which remain the same 
over different evaluation cycles. Totals, averages, percentages and ratios are calculated and 
established for the respective perspectives. This work can be facilitated by a programmed 
Excel sheet, which only requires the input of the raw data. The standardisation of evaluation 



59 Ideally, these data collection procedures are designed to be as easy as possible and are supported by an IT 
system. For example, the data of registered participants might flow into the evaluation analysis directly from the 
administrative database. 

60 In most companies, executives are assessed annually on the achievement of their goals and performance, 
which is then used to calculate the variable share of a Performance Related Pay scheme. 
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data over consecutive cycles is essential in order to attain a longitudinal comparison of 
results. 

The qualitative data is collected in order to complement and explain the quantitative figures. 
Here, standardised summaries exist, as well, in order to ensure that the same feedback is 
available cycle after cycle. Different techniques of qualitative social research can be applied 
to structure the analysis of the data. We suggest the interpretative-reductive content analysis 
of Mayring (1989, 1993) (see subsection 4.2.3.3.2), but other techniques can be applied 
according to the focus and the context of the evaluation. 

Both stages - collection and analysis - should obey one guiding maxim of feasibility. The 
methods used should be easy to understand and to apply by the staff of a corporate 
university. Furthermore, they should not take too much time or too many resources, 
compared to the core activities of running the executive education. The best test for 
feasibility is the continuous use of the evaluation tool in practice. Cumbersome procedures 
are not likely to survive in the business environment of today, especially if they do not yield 
any utility to its users (Patton 1997). In the framework of STEER, we rely on existing data 
and thereby try to leverage synergies across evaluation instruments. The use of categorised 
indicators aims at reducing complexity. An empirical example of the implementation of the 
Strategic Tool for Evaluating Educational Returns is shown in chapter 4. 



3.4.2 Organisational Interfaces of the Research 

STEER considers several sources of evaluation data, which are external to a corporate 
university. The rationale is to ensure an independent opinion and objectivity. 

3.4.2. 1 Assessment of Executive Performance 

In most organisations, a wealth of data exists, which tracks the performance and assessment 
of its managers and executives over time. We propose to include parts of the executive 
assessment data in its evaluation. However, an integration of the data requires that the 
evaluating entity, the people of executive education, and the people responsible for the 
performance assessment co-operate and share information. Therefore, an important 
element of the successful implementation of STEER is a careful management of the 
organisational interface to the performance appraisal function. The underlying idea is that 
both functions benefit from the exchange of evaluation data. 
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The human resource function that is responsible for the assessment of executive 
performance tracks the development of executives year after year. Thereupon, it offers 
different Human Resource Development initiatives to executives in order to ensure their 
personal and professional growth. For example, if an executive under-performed in two 
consecutive years, his/her assessment data could indicate that the lack of leadership skills is 
likely to be a possible reason. As a result, the executive might take part in an executive 
programme of a corporate university. Therefore, at the end of the year, the corporate 
university and the performance appraisal function benefit from the evaluation as it reveals the 
effectiveness of executive education. 

The rationale is that parts of the yearly executive assessment data are used to complement 
the internally generated evaluation data. The professional and personal development of 
participants can be compared to that of non-participants - considering control group 
techniques. 



3.4.2.2 Executive Education Stakeholders 

The implementation of STEER requires that important stakeholders of executive education 
are considered in the evaluation. In most cases these stakeholders are situated elsewhere in 
the organisation and must therefore be addressed across internal departmental boundaries. 

The difficulty lies in the successful co-ordination of collecting feedback and evaluation data, 
which is compatible with the remaining evaluation results. A routine of interviews should be 
established, which reviews the work of a corporate university at the end of an evaluation 
cycle. The evaluator is required to keep his/her contacts to key stakeholders updated. This 
review is important for providing another external perspective on the realisation of executive 
education. The stakeholders’ feedback ensures that the programmes’ content and methods 
are aligned to wider organisational needs. 

To conclude the section - the key to a successful implementation of STEER is the co- 
ordination of the different evaluation instruments. Extensive parts of the evaluation can be 
delegated decentrally and supported by IT, but at the end of the day, the different data 
streams must be managed and analysed. This implies that specific care must be taken to 
maintain the network of stakeholders. 
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3.5 Expected Benefits of STEER 



“Training executives must be more effective in selling the value-added 
component of their service. Better communication closes the gap between what 
trainers think needs to be done and what managers think they need to improve 
performance. One of the primary benefits of the movement to measure value is 
the dialogue it creates between trainers and managers as to what are the most 
important priorities. ” 



(Hackett 1997, p. 20) 

The remaining part of the chapter takes a look beyond the implementation and portrays the 
wider benefits of applying STEER. Thereby, the thesis distinguishes between direct benefits 
for the evaluator and wider positive effects for the organisation. 



3.5.1 Individual Benefits for the Evaluating Entity 

The importance of evaluation is emphasised by most writers on training and development (cf. 
Bronner & Schroder 1983). However, few organisations conduct evaluations that are suited 
to their needs (cf. Hackett 1997). STEER aims at providing the right amount of evaluation 
results without becoming too cumbersome, complex and expensive. 

It is understood that the competition for internal funds within organisations is likely to rise. 
Therefore, the demand for quantitative and qualitative information on the value contribution 
of executive education is likely to increase rather than decrease (Carnevale & Schulz 1990). 
Through STEER we aim at providing essential insights on the value creation of executive 
education, which can justify the investment in Human Resource Development. Specifically, 
the Strategic Tool for Evaluating Educational Returns provides the following benefits for a 
corporate university. 

First of all, STEER relies on simple calculations and evaluation procedures. The four 
perspectives contain a selection of indicators that reflect the value creation of executive 
programmes. Most indicators are self-explanatory and can be calculated quickly and easily. 
The raw data is collected through different channels, which are common and well-known in 
social research. 
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Secondly, the approach covers the most important aspects of executive education with its 
four perspectives. The underlying structure offers a balanced assessment of educational 
programmes. The evaluator can be ensured that the evaluation does not drive him into one 
direction, like a uniform focus would. 61 The evaluation is based on a comprehensive 
understanding of the system of education including its wider effects on the organisation and 
its strategy. 

Thirdly, the new evaluation tool generates understandable data that can be communicated 
effectively. STEER comprises a large share of descriptive statistics, which can be 
understood by management. Totals, averages, percentages and ratios are the common 
language of business and we attempt to present the results of STEER in that language. 

Fourthly, STEER remains in a reasonable frame in terms of time and cost resources. A 
selection of indicators per perspective should keep STEER manageable even in busy daily 
operations. A lot of the evaluation data can be derived from existing data and no extensive 
analysis is needed. Standardised procedures ensure that only the necessary data is 
collected. 

Finally, STEER offers flexibility to its users. Within the framework of the four perspectives, 
different combinations of indicators can be tried. Depending on the type and the goals of 
executive education, suitable measures can be selected or defined individually. Thereby, the 
appropriate emphasis can be ensured by the evaluator. 

Overall, the STEER tries to offer a practicable evaluation approach. It attempts to provide the 
evaluator with as many perspectives and data as possible, considering only necessary 
procedural efforts. The practical application of STEER in chapter 4 provides us a feedback 
on the weaknesses of STEER (to be presented in chapter 6). 

As a result, STEER enables a better understanding of the value creation of executive 
education. Thereby, the evaluator should be in a better position to take decisions on the 
programme portfolio and activities. Finally, STEER demonstrates the value of the investment 
in executive education, which should provide a corporate university with a stronger position 
in the wider organisational context. 



For example, an evaluation approach that solely focuses on the right content could run into the danger of 
missing important organisational aspects of training and development. For example, a programme could have 
great topics and teaching methods, but because it does not reach its target group on time, it is of little value to its 
participants. 
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3.5.2 Organisational Benefits for the Sponsor of the Evaluation 

STEER attempts to induce wider organisational benefits as well. Many of the previous 
evaluation approaches are limited in their scope to the training and development department. 
Their focus is on content and teaching techniques without considering the wider strategic 
implications. 

The stakeholder and business perspectives of STEER emphasise the importance of intra- 
organisational integration of executive education. STEER reflects the fact that Human 
Resource Development is not taking place in a vacuum. Instead, executive education can 
represent a vital part in the drafting and implementation of strategy in an organisational 
setting (Lee 1996). The evaluation methodology of STEER supports therefore the 
communication and exchange between the sponsors and the programme organisers. From 
an organisational point of view, the evaluation ensures that executive education can be 
aligned more effectively to the prevailing strategic direction. 

For example, the regular evaluation feedback from sponsors and supervisors enables 
programme organisers to consider important strategic initiatives early for the training and 
development of executives. Sponsors and supervisors are likely to develop a sense of 
ownership for new ideas. On the other hand, the efforts of Human Resource Development 
have the chance to become more transparent and respected. Therefore, an organisation is 
likely to benefit from a better understanding and co-operation of top management and a 
corporate university. The integration of the four perspectives ensures that the learning about 
the value contribution of programmes is more varied and differentiated than before. 

Finally, we expect better decisions from the application of the tool. STEER provides 
structured evaluation data to the staff in executive education. Therefore, valuable executive 
programmes can be more easily distinguished from weaker ones. As a result, executive 
education should improve over time implying a concrete financial advantage to the 
organisation. 

Concluding chapter 3 - we have developed a new tool in order to suit the practical needs of 
real life evaluation. The novelty is the distinction of four evaluation perspectives, each filled 
with indicators that reflect the value contribution of the executive programme. The next 
chapter illustrates an example of a first application of STEER in practice. 
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4 Application in the Field 



“Evaluators must be pragmatic and conduct their research accordingly. ” 

(Rossi etal. 1988, p. 9) 

In chapter 3 we presented the new evaluation approach STEER, which has been developed 
in the framework of our thesis. We explained the conceptional rationale of STEER and 
described its structure and content. In terms of our evaluation research, we have 
acknowledged that the value of an evaluation tool depends on its applicability and its 
usefulness in real life (Patton 1997). Therefore, we have applied and tested STEER in the 
framework of a corporate university. In the following chapter we introduce the reader to the 
field of research and guide him through the practical implementation of STEER. 

In part 4.1 we describe the our field of research - the DaimlerChrysler Corporate 
University (DCU) in its structure of 1999 to 2001. We explore the history and the situation of 
the DCU and outline its executive education activities. In part 4.2 we present the 
comparative study - the principal piece of empirical research of our thesis. Based on the 
insights and considerations of our Theoretical Frame, we have applied and tested the new 
evaluation approach STEER to the executive programmes of DCU. Thereby, we have used 
STEER to compare the value contribution of internal, customised programmes of the DCU 
with similar external, non-customised programmes of the open market. 62 



4.1 Description of the Field of Research - The DaimlerChrysler Corporate 
University (1 999-2001 ) 63 

“We understand the role of the DCU as a strategic instrument for the support of 
the ongoing development of our organisation. " 

(Gunther Fleig, Board Member for Human Resources and Labour Relations 

Director at the DaimlerChrysler AG) 



62 We discuss the characteristics of the two programme types and their participants in detail in section 5.1.1, 
which presents the results of the study. 

63 This is the period when our research took place. 
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The study presented here has been sponsored by the DCU. The underlying aim has been to 
develop a new way of evaluating DCU’s executive management development programmes. 
We were chosen to carry out the evaluation project at the DCU and started working in the 
field in the summer of 1999. Throughout the rest of 1999 and the first half of 2000 the 
evaluation tool was developed and tested. From the early summer of 2000 the field work 
then concentrated on advancing the academic research of the thesis. The specific research 
focus was set to comparing the customised DCU programmes to similar non-customised 
programmes of the open market (cf. section 2.2.2). Thereby, the direct access to the field 
and the experience gained through the prior project work served as the basis for the 
empirical study. Throughout the entire work in the field the research has been supervised by 
the International Institute for Management Development (IMD) and the Ecole des Hautes 
Etudes Commerciales (HEC) at the University of Lausanne in Switzerland. 

In the following description of the field of research we focus on four areas in order to outline 
the characteristics of the DCU. These four areas include all aspects of the corporate 
university and provide therefore a good overview of the field. The four areas are: 

• The history of DCU - How did the DCU develop over time? 

• The rationale and organisational role - What is the purpose of the DCU? 

• The stakeholders involved - How is the DCU organised in terms of staff, suppliers and 
customers? 

• The learning architecture - What are the specific activities of the DCU? 

In exploring the DCU we applied both primary and secondary research methods. These 
provided the data for the description of the field. 

In terms of primary research methods, we used our experience in the field in form of 
participatory observation (Bortz & Doring 1995). Specifically, our participatory observation 
included the daily project work at the DCU. For the duration of the research we have acted 
as a temporary member of the corporate university. We took part in regular staff meetings, 
project related meetings and internal discussions. Furthermore, we participated in various 
DCU events, in particular several executive education programmes. An important part of the 
primary research has been the regular informal communication and exchange with members 
of staff. Additionally to the participatory observation, we conducted five formal interviews 
with the management of the DCU. 
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Our secondary research consisted of the analysis of internal and external DCU 
presentations and publications. We analysed the secondary material in order to outline the 
development of the ideas and concepts at the different stages of the DCU. These documents 
included presentations given to the board of management of DaimlerChrysler, which explain 
the rationale and architecture of the DCU. Furthermore, we reviewed documents that 
describe the operational processes and the strategic orientation of the DCU. 



4.1.1 The History of the DCU 

“DaimlerChrysler established a Corporate University in response to its 
decentralised and global business structure. Its purpose is to communicate and 
implement strategy and to foster the development of strategic executive 
leadership within the organisation. DCU contributes to the increase in corporate 
value through its strategic content and its integration efforts . " 

(Gunther Fleig, Board Member for Human Resources and Labour Relations 

Director at the DaimlerChrysler AG) 

The purpose and nature of DCU is best understood in the context of the corporate 
development of DaimlerChrysler. Hence, we start with a brief description of the company. 

DaimlerChrysler is a global automotive company that was created by the merger of the 
German Daimler-Benz AG and the U.S.-American Chrysler Corporation in 1998. Both 
Daimler and Chrysler possess a long industrial history in the car manufacturing business that 
began 115 years ago with the invention of the very first automobile. Daimler-Benz has its 
roots in south-west of Germany, Chrysler in the North of the United States. 

Daimler-Benz was formed by the merger of the Daimler-Motoren-Gesellschaft and the Benz 
& Cie. in 1926. For most of its history, Daimler-Benz has been a luxury car and premium 
truck producer - represented by the Mercedes-Benz brand. In the 1980s, Daimler-Benz 
started to pursue a strategy of building a technology conglomerate. The company extended 
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its business into the aerospace, rail and consumer goods industry. In 1997, the management 
decided to refocus on the automotive business and to divest all other businesses. 64 

The Chrysler Corporation was founded in 1925. Over the years, the company rose to 
become one of the top three automotive players in the United States - combining a number 
of brands like Chrysler, Jeep and Dodge. In its history, Chrysler focused primarily on the 
production of automobiles despite a brief engagement in the U.S. missile and space 
programmes in the 1950s and 1960s. Its recent history has been a turbulent one 
characterised by almost bankruptcy in the early 1980s and successful turnaround plans that 
made it one of the most profitable car producers of the 1990s. 

At the end of the year 2000 DaimlerChrysler employed 416,000 employees. Today the 
company is represented in 40 countries. In 2000 its annual sales amounted to EURO 162.4 
billion, resulting in a net profit of EURO 4.4 billion. The corporate headquarters of 
DaimlerChrysler are in Stuttgart, Baden-Wurttemberg, Germany. However, due to the 
merger, another important administrative centre is located in Auburn Hills, Detroit, Michigan, 
USA. 

The initial idea of the corporate university stretches back to the time of the Daimler-Benz 
holding company 65 in the early 1990s. The former CEO of Mercedes-Benz, Mr. Werner, and 
the respective board member for personnel, Mr. Tropitzsch, initiated a project team to 
develop an "instrument of strategic education” (quote from an interview with a Senior 
Manager). According to an anecdote, Mr. Werner saw the need for developing the strategic 
thinking of his executives when he discussed the new idea of the A-Class. 66 He found little 
understanding and support for the A-Class among his executives - few were able to share 
the strategic vision of the new product. 67 As a result, he commissioned an educational 
institution - later named the corporate university - to enhance the strategic thinking of the 
executives in the company, i.e. their strategy process capability (cf. 2.2. 1.1). The new 
institution for executive education was meant to improve and align the strategy process 
capability of the company’s executives. 



64 The process of divesting other businesses has been ongoing during the research. Although most of the larger 
subsidiaries like Dasa (aerospace and defence sector) or Adtranz (rail sector) had been sold since 1997. 

65 The holding structure implied that Daimler-Benz combined independent subsidiary companies like Mercedes- 
Benz, Debis (service sector) or Dasa (aerospace and defence sector) 

66 The A-Class was to be the first Mercedes-Benz model in the premium small car segment. Until that date, 
Mercedes-Benz had focused on the large and luxury car segment. 

67 Mr. Werner saw the need for the A-Class in order to extend the product portfolio and to become a full-range 
automotive company. 
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The ideas of the corporate university were developed further under the supervision of Mr. 
Schrempp, the new CEO of the Daimler-Benz AG, after the dissolution of the holding 
company and the internal merger of Daimler-Benz and Mercedes-Benz. In the summer of 
1997 the board of management gave the go-ahead for the Daimler-Benz Corporate 
University, which came into full operation in 1998. 

A first core team conceived the structures and processes of the corporate university in 1997. 
As we see in subsection 4. 1.2.4, the core team implemented the original idea of the strategic 
instrument, but also created a broad activity portfolio including executive education, 
leadership development and knowledge management. Subsequently, the Daimler-Benz 
Corporate University was staffed and went into full operation in late 1998. At the time it 
consisted of 14 permanent members and was situated in the Mohringen headquarters. The 
creation of the corporate university thus represented an enlargement of the existing 
corporate portfolio of education activities. The corporate university did not substitute, but 
complemented the more technical training departments within the company. Therefore, the 
initial challenges of the Daimler-Benz Corporate University were to set up its activities, but 
also to integrate its processes in the existing structures of the human resource area of the 
organisation. 

The merger of Daimler-Benz and Chrysler in late 1998 caused a number of internal changes 
for the corporate university. The original purpose and structure remained with a new focus on 
post-merger integration. Chrysler had not possessed a corporate university as such before 
the merger, but had its own experience and expertise in the field of executive development. 
These expertise and competencies were united with the Daimler-Benz Corporate University 
to form the DaimlerChrysler Corporate University (DCU). Thereby, the DCU experienced 
a substantial turnover in personnel with respect of its founding members. 68 As a result of the 
merger the DCU was not only staffed from the Daimler, but now also from the Chrysler side. 
An early departure of the first Dean made the change in personnel visible. The new Dean 
and a number of managers were brought in from Chrysler and a second DCU location in 
Auburn Hills was established. 

The DCU grew in numbers due to a more general change in the area of Executive 
Management Development (EMD) which is the umbrella division to which the corporate 



68 During the post-merger integration process ca. 65 percent of the DCU staff changed their responsibility within 
DaimlerChrysler. Some employees left the company for good. 
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university belongs. 69 In the early summer of 2001 the DCU comprised ca. 25 members. 
Reviewing the history of the DCU on the whole we can discern a very dynamic development. 
The corporate university was created with the support from top management and therefore a 
lot of enthusiasm existed in the first months. Later the DCU experienced significant changes 
as a result of the post-merger integration process. 



4.1.2 The Rationale and Organisational Role 

The DCU represents a strategic instrument for the alignment and the implementation of 
corporate strategy. This principal purpose has remained constant throughout the existence of 
the corporate university. The continuity can be seen by comparing the purpose of the DCU 
as formulated at the go-ahead decision in 1997 with the most recent board of management 
presentation in 2000. 

“The Daimler-Benz Akademie [later DCU] supports the group’s strategic 
orientation and intensifies management professionalisation in key competencies. ’’ 

(Decision of the Board of Management, Internal DCU Document 1997) 

“Positioning: DCU as strategic instrument to enable implementation of the 
strategic goals of the company. ” 

(Presentation to the Board of Management, Internal DCU Document 2000) 

The underlying rationale of the positioning of the DCU becomes more transparent when we 
take a look at its organisational role. Figure 4.1 (on the next page) illustrates the strategic 
alignment and integration of the DCU. Applying the theory about Strategic Human Resource 
Development (SHRD) (cf. 2. 2. 1.2), we can classify the DCU as an example of SHRD. 

The corporate university is a strategic instrument that assists in pursuing the objectives of an 
organisation (Garavan 1991). In the terminology of our theoretical framework, the DCU 
therefore plays both an aligning and an advancing role (cf. subsection 2.2. 1.2). It has an 
aligning role because it develops and aligns the skills and knowledge that are required for the 
implementation of a given strategy. Furthermore, the DCU plays an advancing role as well, 

69 Executive Management Development (EMD) comprises recruiting, staffing and supporting functions for all 
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since it addresses the improvement of the strategy-making and implementation skills of the 
members of the organisation (see quote in the beginning of part 1.1). In figure 4.1, the two 
points Thinking strategically and establishing direction and Leading change 
demonstrate the additional focus on strategy process capability as defined in chapter 2. 
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Figure 4.1 : The Role of DCU in the Strategy Process of DaimlerChrysler 

Source: DCU 2000 



The specific activities of the DCU are derived from its organisational role and are stated in its 
value proposition in figure 4.2 (on the next page). These activities refer to the DCU portfolio 
on the whole and show how the corporate university can enhance the competitive advantage 
of DaimlerChrysler. 



In addition to the support of strategic initiatives and the development of critical capabilities , 
figure 4.2 illustrates three more purposes. The DCU serves as an instrument for integrating 
the organisation, for sharing knowledge and best practices and for providing a platform for 
discussion. As a consequence, the DCU does not focus on executive programmes alone, but 
possesses a broader activity portfolio - as we see below. 



executives in the company and includes the corporate university as well. 
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The abstract graphical illustrations, written in consulting speak 70 , provide us with a broad 
understanding of the rationale and role of the DCU. In the next two subsections we describe 
the actual work of the DCU in more detail by looking at the people involved and the 
programmes and events offered. These give us a more practical grounding of DCU. 



4.1.3 The Stakeholders Involved 

The DCU is a virtual organisation, which relies on a network of staff, internal stakeholders, 
external suppliers and customers. In practice this implies that there exists a permanent core 
team located both in Stuttgart and Auburn Hills. This team comprises ca. 25 members and is 
managed like a department by a Dean and four Senior Managers. It has a functional 
structure, which represents the learning architecture of the DCU (see section 4.1.4). The 
core team decides on the content and the form of all DCU activities through a constant 
dialogue with internal stakeholders, who provide impulses and directions. Subsequently, the 
team plans and devises the programmes and events in co-operation with external partners, 

70 Consulting speak refers to the highly abstract and shortened language used by consultants and senior 
management. It allows that complex problems can be discussed on a global, superficial level by omitting the 
technical detail. The disadvantage of consulting speak is that it provides little concrete information and insight to 
the reader and that therefore it runs into danger of being to detached from reality. 
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who then provide the resources and deliver the programmes and events. Overall, the key 
competencies of the DCU is to set the right content and to select appropriate methods. 
Furthermore, it co-ordinates the delivery by external partners in order to ensure a high 
degree of customisation. 

The DCU includes participant management and administration of its programmes and events 
as well. The application procedure varies according to the type of programme. In the case of 
open-enrolment participants apply directly to the DCU for a place. Nomination implies that 
participants are selected by their respective personnel managers. Quotas exist for many 
programmes, which define the shares of participants from different business units. All 
programmes are free of charge, i.e. borne centrally by the DCU, only travel cost are borne by 
the participants. Overall, the DCU is funded by the office of the CEO and run as a cost 
centre. 
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Figure 4.3: The Virtual Organisation of DCU 

Source: DCU 1999 



The internal stakeholder network includes the top management of the company as well as 
important internal experts on a given topic. Their role is twofold. On the one hand, they 
initiate new content and provide new impulses - in particular with respect to corporate 
strategy. For example, the board of management would emphasise a new direction in the 
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corporate strategy and therefore the DCU would align its content accordingly. On the other 
hand, the stakeholders provide direct input to programmes and events as internal speakers 
on a given topic. For example, Mr. Schrempp has been speaking to executives during DCU’s 
executive programmes. 

The external supplier network consists of selected experts and professors who deliver the 
customised content in form of an event or programme. The DCU works together with experts 
for specific projects and partners with business schools for larger customised interventions, 
as for example an executive programme. 

As a result of the virtual organisation structure, the DCU does not possess any teaching 
location in the form of a building or campus. The realisation of its programmes and events 
are outsourced to existing DaimlerChrysler training centres, to the facilities of the respective 
business schools or to hotels and resorts. 

The DCU focuses on a well defined target group within the organisation, which differs from 
the all-employees-approaches of other corporate universities (cf. Motorola University). The 
target group consists of the company’s middle management (cf. definition in subsection 
2. 2. 1.1), i.e. the Vice Presidents (internal designation: El), the Directors (E2) and the Senior 
Managers (E3) of DaimlerChrysler only. 




Figure 4.4: The Internal Stakeholders and Customers of DCU 

Source: DCU 1998 
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To summarise - the DCU consists of a small team of specialised staff. It does not posses its 
own teaching facilities - all programmes and events are outsourced to internal providers or 
external partners. The target group comprises the top 7,000 executives of the company, with 
the exception of the top management. 



4.1.4 The Learning Architecture 

The novelty of the DCU concept relies on a new combination of different educational 
activities. The core part have been executive education programmes, but other functions 
have been combined in the DCU portfolio as well. Therefore, we speak of an integrated 
learning architecture (Deiser 1999). In the time of 1999 to 2001 the DCU portfolio consisted 
of four pillars - which can be seen in figure 4.5 - each led by a Senior Manager. 



Although our research focuses on the management programmes offered by Executive 
Education, we present all four pillars briefly in order to ensure a holistic understanding of 
DCU. 
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Figure 4.5: The Learning Architecture of DCU 

Source: DCU 2000 
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Strategy Transfer offers customised events based on the idea of dialogue and exchange. Its 
primary role is to communicate important strategic themes into the organisation in a “fast, 
first-hand and unfiltered way” (brochure Strategy Transfer 2000). At the same time it provides 
executives with the opportunity to feed back their perspectives and experiences to top 
management. For example, the Strategy Transfer team staged a one-day awareness forum 
about e-business for executives - bringing together internal and external experts and the 
responsible board members for the topic. These experts presented and discussed the 
dimension and relevance of e-business for the company in front of 250 participating 
executives. 

The overall aim of Strategy Transfer is to achieve alignment of corporate management 
resources in order to accelerate the exchange of strategic thinking and the generation of new 
ideas within DaimlerChrysler. The events of Strategy Transfer are conceived and realised 
internally - due to the strategic nature of the events - and take place at internal and external 
conference locations. 

Leadership Development focuses on building executive leadership capability within the 
organisation. It provides customised programmes of one to two weeks, which are compulsory 
for executives, who have been nominated for a higher management level. Its purpose is to 
prepare the manager for his/her new leadership role. For example, the DaimlerChrysler 
Seminar prepares all Junior Managers, who have been nominated to become a Senior 
Manager. The seminar provides a breadth of leadership skills including administrative 
knowledge as well as management expertise. The Leadership Development programmes are 
designed, customised and realised in co-operation with external companies specialised on 
executive education. They take place at internal training centres or external hotels. 

Executive Education addresses a more global spectrum of business skills including a 
strong focus on teaching strategic thinking. Its programmes aim at enhancing the general 
management competencies and intend specifically to build the strategy process capability of 
DaimlerChrysler executives. These programmes of one or two weeks are conceived and 
customised by the DCU in partnership with leading business schools, who run the 
programme at their campus. For example, the programme Managing Global Opportunities 
teaches cutting edge management knowledge and strategy concepts. It is designed for Vice 
Presidents and provided by the Harvard Business School. Like most of the Executive 
Education programmes, the Harvard programme focuses on practical learning through real- 
life case studies. The three types of programmes, which form the basis for the empirical 
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research, are supplied by Harvard, INSEAD and the IMD. The characteristics of their 
programmes are outlined in subsection 5.1.1 and their content is described in appendix 8.1 1 . 

The pillar of Executive Education has been extended to include programmes that foster 
cross-cultural competence. These cross-cultural programmes have remained a separate 
block from business school education. 71 The traditional area of Executive Education serves 
as the basis for the application of STEER in the empirical research of this thesis (please see 
part 4.2). 

The fourth pillar comprises the Knowledge Management activities of the DCU. The 
corporate university represents the competence centre for knowledge management within 
DaimlerChrysler. Its purpose is to leverage knowledge assets across internal organisational 
boundaries and to facilitate the sharing of best practices. For example, one priority of 
Knowledge Management is the implementation and maintenance of Communities of Practice 
on the corporate and the business unit level. Communities of Practices are informal groups 
of experts with a common concern that communicate and exchange best practices within 
their group across organisational structures. 72 Furthermore, this part of the DCU organises 
business simulations, planning games, discussion forums and inter-company exchanges with 
respect to knowledge management. 

Knowledge Management is also responsible for the DCU-Online - a new intranet-based 
multi-media platform of the DCU. The purpose of the DCU-Online is to support all activities of 
the corporate university - providing the latest information on the DCU curriculum, preparing 
for and following up of programmes, giving online-support to Communities of Practice and 
offering learning modules on strategically important topics. The DCU-Online can be entered 
from all DaimlerChrysler locations world-wide, but limits its access to the specific target 
group of the corporate university. Furthermore, the platform supports alumni activities and 
fosters the intra-company networking among executives. 

To conclude - the DCU represents a new human resource development institution within 
DaimlerChrysler, which has been created in addition to existing training departments. It 
serves as a strategic instrument for the organisation in a dual manner. On the one hand, it 



71 The cross-cultural competence centre offers short seminars on doing business in selected foreign cultures. 
These programmes are designed foremost for the group of expatriates and executives, who deal with these 
countries frequently. 

72 For example, due to a reorganisation all the brake engineers of Chrysler have been allocated to different car 
models and are therefore geographically separated from each other (in the former functional structure they had 
been all working together). The Community of Practice ‘Chrysler Brakes’ has re-established meetings of these 
engineers in order to ensure the continuous exchange of knowledge and experience. 
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transmits and shares strategically relevant content in form of topics and knowledge. On the 
other hand, it builds leadership and general management skills with a strong focus on 
strategic thinking, i.e. strategy process capability. In particular the latter purpose caused our 
interest in the DCU and led to the research, which is presented in the following part. 



4.2 Comparative Study - Customised vs. Non-Customised Executive 
Programmes 

“The question whether training can enhance the strategic thinking ability of senior 
managers is difficult to answer due to the lack of research in this area. Clearly, 
more research is needed in the underrated study of strategic thinking .” 

(Bonn 2001, p. 69) 

Having presented the general field of the research, we now take a closer look at the 
empirical study conducted in the framework of our thesis. The overarching idea has been to 
apply and test the new evaluation approach STEER in order to assess the value 
contribution of customised executive programmes to executives’ strategy process capability. 

In part 4.2 we outline the methods used in the implementation of STEER. First of all, in 
section 4.2.1 we review the specific rationale for conducting the research. Then, in section 

4.2.2 we present the research questions and the hypothesis that have guided us through the 
comparative study. We present the research methodology in section 4.2.3 including technical 
details like sampling and the research techniques used. Finally, in section 4.2.3 we share the 
observations in the implementation process (the results are presented in chapter 5). 



4.2.1 The Rationale of the Comparative Study 

We choose the management development programmes of the Executive Education area of 
DCU to apply STEER practically. In particular, we decided to compare the customised 
programmes of Executive Education with similar non-customised programmes on the open 
market. There were several reasons for doing so. 
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First of all, one focus of the research has been the strategy process capability of executives 
and the way that it can be built and developed. The customised programmes of Executive 
Education possess a strong focus on fostering the very skills, knowledge and mentality of 
strategic thinking, i.e. strategy process capability as defined by us in the Theoretical Frame. 
The executive programmes of DCU therefore offer a good opportunity to investigate the 
value of teaching strategy process capability. 

Secondly, the practical constellation of the project work in the field created a unique access 
to the participants, stakeholders and staff of the DCU and its executive programmes. By 
working directly in the field we received a well-founded understanding of the practical 
implications and underlying dynamics of executive education. From an academic 
perspective, the work allowed us to obtain new insights of the processes of management 
development programmes. As a member of DCU we were able to conduct a more efficient 
and effective research compared to a person from the outside. 

We have been aware of the influence and bias arising from our engagement in the field. 
However, the academic supervision by an external university and the constant feedback from 
non-involved people helped us to remain objective and independent. 

The third reason has been the opportunity to apply the new evaluation approach in a 
real-life environment. The access to the field favoured an application of STEER as conceived 
in chapter 3. We could therefore test the new approach in the very situation for which it has 
been designed. As a result, the thesis does not only contribute new theoretical ideas to the 
evaluation debate, but, at the same time, provides practical experiences of the application of 
a new tool as well. We were able to make inferences on the quality of the tool itself. 

Finally, we decided to apply STEER not only to the customised programmes of Executive 
Education, but to non-customised programmes as well in order to derive a comparison. The 
main rationale for doing so was the constrained time frame of the research, which did not 
allow a longitudinal application of the new approach in consecutive years. Therefore, a 
cross-sectional testing was conceived. The dual application is important because it assists 
in filling the measures of STEER with meaning and practical relevance. The results of each 
application can be referenced against each other. By comparing the two groups of 
programmes (customised vs. non-customised), STEER produces not only information on the 
general value creation, but provides concrete insights on the strengths and weaknesses of 
customisation. Thereby, the non-customised programmes also serve as a kind of control 
group to the application of STEER to the customised programmes. 
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The comparison of customised to non-customised programmes is of interest from an 
academic perspective as well. The specific constellation of the research - in particular the 
chance to keep important variables constant in the comparison (like the industry and the 
company, see subsection 4.2. 3. 1.1) - enables an investigation in the nature and the 
dynamics of customisation of executive development programmes. The results of the study 
provide evidence on how customised programmes, in contrast to non-customised 
programmes, build and develop strategy process capability. We thus provide a better 
understanding of the strategic value of customised executive education. The research 
gives new insights to the recurring question of customising or outsourcing the executive 
education. 



4.2.2 The Research Questions and Null Hypothesis of the Comparative Study 

We have formulated the following research questions for the comparative study in order to 
structure the empirical research further. These questions assist us in drawing conclusions 
from the application of the new evaluation approach. 

CS1: 73 Do the executive education programmes of the DCU contribute to the strategy 
process capability of their participants? 

CS2: What are similarities of customised and non-customised executive education 
programmes? 

CS3: What are differences of customised and non-customised executive education 
programmes? 

CS4: Do customised and non-customised programmes differ in their potential to build 
individual strategy process capability? 

CS5: Can the value contribution of either type of programme be measured in a valid and 
reliable way by applying STEER? 

The application of STEER serves us to tackle the aspect of customisation. In section 2.2.2 
we have seen that contradictory opinions exists on the value of customisation. Some 
authors even doubt whether there is a real difference between customised and non- 

73 CS = Comparative Study 
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customised executive education. Hence we formulate a general null hypothesis for our 
comparative study. 

Customised and non-customised programmes do not differ and produce 
the same value to the individual and the organisation. 

Through STEER we hope to find evidence that either confirms or rejects the null 
hypothesis. Furthermore, STEER should provide more detailed answers to the five research 
questions above. 



4.2.3 The Research Methodology of the Comparative Study 

In the following subsection we outline how STEER has been applied to a real-life situation. 
First, in subsection 4.2.3. 1 we take a look at the sampling procedures chosen. We describe 
the total and sub-populations and the two samples taken. Furthermore, we outline the 
specific sampling techniques applied. In subsection 4.2. 3.2 we outline how the different 
evaluation instruments of STEER have been applied. 

For the remainder of the empirical study we use the following labels: 

C connotes the customised programmes (“C” = customised) 

NC stands for the non-customised, external programmes (“NC” = non-customised) 



4.2.3.1 Sampling Procedures 

In this subsection we review how we defined the two groups of interest - the customised 
DCU programmes on the one hand and the non-customised external programmes on the 
other. First we take a look at the characteristics of the different populations. Then we discuss 
the sampling techniques that we have used. 

4.2.3. 1.1 Characteristics of the Different Populations 

Figure 4.6 (on the next page) illustrates the overall structure of the relevant populations and 
samples for the empirical research. We explain the main elements briefly in the following 
subsection. 
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Figure 4.6: Overview of the Population(s) and the Samples of the Empirical Research 



The size and diversity of the Total Population favours the underlying sampling idea as 
described in figure 4.6. The Total Population includes a sub-population of those executives 
that have participated in the three DCU programmes of interest. These executives represent 
our C-Population. Furthermore, the Total Population also comprises a sub-population of 
those executives that have taken part in external programmes - run by the same three 
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business schools that deliver the three programmes for the DCU. 74 These executives form 
our NC-Population, which unfortunately proved to be relatively small. 75 

The underlying logic of our distinction is to keep as many variables as possible constant in 
order to isolate the effects of customisation. The fact that the two sub-populations come from 
one Total Population enables us to hold the following variables fixed (see table 4.1). 



Variables 


C-Population 


NC-Population 


Fixed variables (held constant) 


Teaching format Face-to-face 

Size: 25 to 40 participants 

Teaching methodology: Predominantly case-study 

Type : On campus for several days 

General orientation: General management with focus on 

strategic thinking and on implementing change 

Providers: IMD, INSEAD, Harvard Business School 


Isolated variables (changing) 


Customised content 
Closed-enrolment 
Only selected 
DaimlerChrysler 
executives 


Non-customised content 
Open-enrolment 
Executives’ diversity 
(different firms and 
industries) 



Table 4.1 : Overview of the Fixed and the Changing Variables 



All members of the Total Population work for the same company - most of them in the 
automotive sector. It is likely that the majority have an engineering background. Furthermore, 
we can say for certain that over 90 percent of them are male and aged 35 to 60. Therefore, 
in terms of their business background, these 7,000 DaimlerChrysler executives represent a 

relatively homogenous Total Population. 

With respect to the two sub-populations C and NC, we can say that all members have 
participated in similar style executive programmes run by the faculty of the three business 
schools (Harvard, INSEAD and IMD). The distribution of participants are fairly similar in 
both sub-populations with respect to the three business schools. 

In the C-Population the shares are Harvard 39 percent (166 participants), INSEAD 41 
percent (169 participants) and IMD 20 percent (83 participants). The share of the NC- 
Population are Harvard 22 percent (9 participants), INSEAD 39 percent (16 participants) 
and IMD 39 percent (16 participants). 



We discuss the characteristics of the external non-customised programmes in section 5.1.1. 

75 On the grounds of the overall size of the Total Population, we expected a larger NC-Population when devising 
the research structure. 
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None of the programmes pre-selected their participants with respect to functions. Hence, we 
can assume a random distribution of management functions in the two samples. 

All programmes are paid for by the company. However, in the case of the customised 
programmes, the DCU covers the complete programme and accommodation fees. Only the 
travel expenses are borne by the respective departments or business units of the 
participants. In the case of the non-customised programmes, all fees and costs are usually 
borne by the departments/business units of the participants. 

4.2.3. 1.2 Sampling Techniques 

With respect to the two sub-populations we had to apply different sampling and selection 
approaches. For the C-Population we chose a stratified random sampling technique and 
for the NC-Population we were able to conduct a census. The reason for this is that the NC- 
Population was substantially smaller (N N c=41) than the C-Population (N c =418). Therefore, 
the characteristics of the smaller NC-Population have influenced the stratified random 
sampling of the larger C-Population. We illustrate each one in turn. 

The NC-Population consists of only 41 executives as their participations in the external 
programmes have been organised individually within the company. This implies that each 
executive has taken part in an external business school programme on his own - for 
reasons, which we do not know so far (but which we find out in chapter 5). In a medium-sized 
company these individual participations would amount only to a handful of cases. However, 
the large Total Population of executives within DaimlerChrysler and its flexible training 
policies have fostered the participation of many executives in external management 
development programmes. As a result, the NC-Population of DaimlerChrysler comprises 41 
executives. We would like to point out the significance of this number - these 41 executives 
were sent independently from each other to similar management development 
programmes at the same three partner business schools of DCU in the last five years. On 
the grounds of its small size we conducted a census of the NC-Population (Bortz & Doring 
1995). 

In terms of the C-Population we conducted a stratified random sample (Chisnall 1992; 
Bortz & Doring 1995) in order to render the two samples compatible. We have been able to 
conduct stratification because we possessed a list of the population including important 
characteristics of its members (as for example which business school/programme they 
attended and their executive level). We chose a stratified random sampling technique since 
we were unable to interview all of the 418 executives in our study due to time and resource 
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constraints. Moreover, stratified random sampling enables more accurate results compared 
to simple random sampling (Bortz & Doring 1995). We took a sample of 80 executives (ca. 
20 percent of the C-Population). Together with the 41 interviews of the NC-Sample, these 
amounted to over 120 scheduled interviews in total (which we considered the largest number 
of interviews manageable in the framework of our research). 76 

From the list of participants, we decided to stratify the C-Sample according to the three 
business schools. These business school strata correlate with the executive level of 
participants. For example, the target group for the Harvard programme are the Vice 
Presidents (El) of the company. Consequently, by separating the participants of the Harvard 
programme, we selected the Vice Presidents of the C-Population as well. 

The stratified random sampling enabled us to determine a compatible C-Sample to the NC- 
Sample. The overall distribution of participants of three business schools were similar in both 
populations. We determined the shares of the three business schools of the C-Population 
and weighted the strata of the C-Sample accordingly. Then we sampled the respective strata 
at random in order to select the C-Sample. For example, the share of Harvard participants in 
the C-Population is 31 percent. Accordingly, we decided to include a proportionate number of 
Harvard participants in the C-Sample - 31 percent of 80 amounts to 25 executives to be 
interviewed. 

As we see in chapter 5, the actual shares of the executives interviewed in the study vary 
slightly from the pre-scheduled figures. The distortion is caused by executives being 
unavailable for the telephone interviews. In general however, we achieved a relatively high 
response rate compared and thus the variation remains small (cf. subsection 5. 1.1.1). 

Having described the underlying sampling procedure, we now take a look at how the different 
evaluation instruments of STEER’S four perspectives have been applied to the two samples. 



4.2.3.2 Evaluation Procedures 

The depiction of the evaluation procedure assists us in understanding the practical 
methodology of the empirical research. We have applied the structure and the instruments of 



A larger size would have taken too long to conduct and analyse - given the time and resources available to us. 
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STEER to both samples according to the conceptional guidelines of chapter 3. Hence, we 
review the respective application of STEER to each sample. 

In the evaluation we encountered several complications in the application process, which 
forced us to alter the planned research instruments. These complications were caused by the 
nature of the empirical research. On the one hand, we experienced difficulties in comparing 
the customised with non-customised programmes. Some of STEER’s evaluation ideas could 
not be applied to the external programmes as we see below. On the other hand, we 
accounted administrative circumstances within the organisation, which forced us to alter the 
procedures of STEER to some extent. However, we were able to gather and analyse the 
majority of evaluation data and thus conduct the comparison successfully. 

4.2.3.2.1 Application of STEER to the C-Sample - The Customised DCU 
Programmes 

In chapter 3 we outlined that a set of different research instruments provide the evaluation 
data for the four perspectives of STEER. Therefore, we illustrate the practical application of 
each evaluation instrument in turn. All forms and questionnaires can be found in appendix 
8 . 10 . 

We suggested that STEER starts the evaluation cycle with a request form for all potential 
participants and their supervisors at the beginning of the year, for example in line with the 
yearly appraisal round meeting. However, the actual application of the request form proved 
to be complex. At the time of the implementation of STEER, DaimlerChrysler was in the 
middle of changing its appraisal procedures, which implied additional meetings and forms in 
comparison to the previous system. The request form was therefore put on hold for another 
year (until the end of 2001), and we had to devise an alternative. Subsequently, we included 
several questions in the follow-up interviews, which reviewed the initial motivation and 
purpose of the executive’s participation. 

In line with STEER we designed a new standardised ‘happy sheet’, i.e. end-of-programme 
questionnaire - a one page, easy to administer feedback form comprising mostly closed 
rating questions. The sheet has been attached to the existing feedback questionnaires of the 
business schools in order to apply one common, standardised evaluation sheet across 
the three programmes. Previously no standardised sheet existed and therefore no 
comparable evaluation data existed for the DCU programmes. The DCU depended on the 
evaluation data of the business schools and there was little transparency and comparability 
between programmes. The new sheet thus concentrates on a selection of standardised 



166 




questions in addition to the evaluation questions of the respective business schools. For 
example, the business schools’ evaluation sheets ask the participants to rate the individual 
sessions and professors, whereas the standardised sheet reviews the learning with respect 
to the leadership criteria of DaimlerChrysler. In the framework of STEER the ‘happy sheets’ 
were distributed to the participants of the three business schools programmes and filled in by 
them at the end of the programme. With all participants in the class-room, our ‘happy sheets’ 
attained response rates of 90 to 100 percent in the DCU programmes. 

One of the principal sources of evaluation data were the follow-up telephone interviews 
with former participants. Before the application of STEER no follow-up evaluation took place 
within the DCU. We decided to conduct the telephone interviews instead of mail or on-line 
questionnaires for a number of reasons. 

Experiences with mail/email questionnaires have shown that they achieve a low response 
rate. For example, during the design phase of STEER we piloted an email questionnaire and 
attained a response rate of only 21 percent despite two reminders (19 sent out, 4 returned). 
On-line questionnaires seem to face even greater problems. An online survey conducted by 
the DCU in the framework of the development of DCU-Online had a response rate of less 
than 2 percent (500 sent out, 7 returned). The reason for these low response rates can be 
found in the nature of the target group. The people in the target group are busy executives 
including plant managers and heads of business units. The usual mail or on-line 
questionnaire is likely to ‘drown’ in the daily work load of these executives. Considering the 
small size of the NC-Population for example, a response rate of even 25 percent would result 
in only 10 questionnaires returned, which would make any statistical inferences impossible. 
Telephone interviews ensure a substantially higher response rate. They are more efficient 
(faster) and more effective (more detailed answers). Therefore, we applied the following 
procedure to our two samples. 

Our C-Sample consisted of 80 former participants, whose names and contact details were 
available from the participant administration within the DCU. As a first step we contacted all 
80 executives by a personalised email - announcing the evaluation study and explaining the 
background and intention of it. A few days later we called the executives’ secretaries in order 
to ask whether the person would be willing to take part in the telephone interview and, if yes, 
to schedule a suitable date and time for them. Once a telephone interview was scheduled - 
we would usually receive a slot of 15 to 30 minutes of the executive’s time - we provided a 
copy of the interview questions to the executive (before the interview) so that he/she could 
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prepare himself/herself. At the scheduled time we would call up the executive and conduct 
the interview. 

We consider this as the most efficient way because the questions were read out to them 
while they could follow on their copy of the survey. The questions were formulated and read 
out in English or in German (the translation was tested on consonance and meaning). 
Responses were captured in the respective languages as well. The telephone interviews 
provided relatively detailed answers since the executives did not have to reply in writing, but 
could reply orally and thereby quicker. The responses were captured by the interviewer 
directly on a pre-printed form. Furthermore, the telephone follow-up interviews allowed that 
executives all over Europe, in the USA, South and Middle America, Africa, Asia and Australia 
could be interviewed with respect to their participation in the DCU programmes. This would 
not have been possible with face-to-face interviews. Our research strategy paid off and we 
achieved a response rate of 90 percent for the telephone follow-up interviews (we were 
able to interview 72 out of the 80 in the C-Sample). We conducted these 72 interviews in two 
batches of 40. After the first batch, we were able to improve the questionnaire again (once 
more after the initial piloting). We were able to add nine new questions in the second batch. 77 

The questionnaire for the telephone interviews comprised 45 questions. We structured the 
questionnaire into 8 administrative questions, 13 open-ended (qualitative) questions and 24 
closed rating questions (quantitative). A copy of the questionnaire is presented in appendix 
8 . 10 . 

In reflecting our methodology, we acknowledge that telephone interviews are not without 
drawbacks. We consider the limitations of our methodology in more detail in chapter 6. 

In terms of the stakeholder interviews we discovered a number of implementation 
difficulties. The interviews with superiors proved to be complex in terms of the resources 
available and the political culture. First of all, we found that the superiors were less 
accessible than the participants. Whereas former participants understood the rationale of the 
interviews - because they themselves took part in the programme - superiors saw little need 
for additional interviews. Secondly, there was little support by the responsible DCU 
managers for approaching superiors due to the organisational changes during the time 
period. A further issue here has been that the participants themselves are high-level 
executives within the company. Therefore, their superiors are often corporate or business- 

77 This implies that we only have 34 responses (out of the second batch of 40 interviews) in the C-sample for the 
respective nine questions. 
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unit board members, who have even less time for an interview. We found it difficult to include 
these high-ranking top managers in the study. 

Interviewing subordinates proved to be problematic in terms of the complexity of work 
councils’ approval, which is needed for non-executive employees. The interviews with the 
subordinates are not central to STEER and therefore we left these instruments for further 
research. A research project with more people and time resources would perhaps be able to 
overcome these administrative obstacles. 

With respect to programme sponsors we scheduled ten qualitative interviews. Unfortunately, 
we were able to conduct only four face-to-face interviews. We found the availability of 
sponsors difficult as well. Six sponsors did not find the time for an interview. For the four 
interviews we used a list of six open questions, which were posed to each sponsor in a 30 to 
45 minute interview (see appendix 8.10). 

Finally, we gathered internal administrative data within the DCU. These include 
programme statistics, participant data and cancellation rates, but also detailed descriptions of 
the programmes, copies of the materials used and a rough overview on costs. 

Throughout the research we conducted informal exchanges of views with the internal and 
external experts, like the manager responsible for the programmes and personnel 
managers. 

We applied and used the above evaluation instruments in the period from May 2000 until 
March 2001. 

4.2.3.2.2 Application of STEER to the NC-Sample - The Non-Customised 
Programmes 

In our thesis we use the term non-customised to describe those programmes, which are 
offered by business schools on the open market. Most institutions run a number of executive 
education programmes and thereby attract executives from all kinds of business and 
functional backgrounds. Our NC-Population focuses on the executive programmes of 
Harvard, INSEAD and IMD since these are the partner schools of DCU. 

The participation of DaimlerChrysIer executives in external programmes is not managed by a 
central corporate training department, neither is it recorded anywhere. Executives are sent 
on an ad-hoc basis by their supervisors or personnel managers, or they organise the 
participation in a programme themselves. Therefore, we could not determine beforehand 
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which of the 7,000 executives would take part in non-customised programmes. As a result, 
we were not able to conduct request forms or ‘happy sheets’ either. 

In terms of the follow-up interviews, we contacted the alumni functions of the respective 
business schools in order to ask for a list of the DaimlerChrysler executives, who had taken 
part in their programmes recently. We received a list of names including the respective dates 
and the programmes visited. Consequently, we had to search for the executives within 
DaimlerChrysler, which proved to be a complex task. Due to the decentralised structure and 
the recent mergers and divestitures there existed no common directory of executives at the 
time. Furthermore, a name proved to be insufficient for the identification of former 
participants within the group of 7,000 executives. In particular, executives located abroad in 
smaller subsidiary companies required a lot of meticulous investigation and a number of 
phone calls to contact them. In this respect, the research access to the field proved to be 
invaluable. It would have been very difficult to identify and contact the executives from 
outside the company, i.e. without our unique access to the field. 

With respect to the follow-up telephone interviews we followed the same procedure as for the 
C-Sample. We were able to achieve a response rate of 83 percent (we interviewed 34 out 
of the 41 executives). 

Due to the experiences in the C-Sample, we decided to refrain from contacting further 
stakeholders. On the one hand, we experienced the same organisational and partly political 
complications as in the C-Sample. On the other hand, the underlying purpose of the NC- 
Sample has been to create a point of comparison to the C-Sample. The fact, that we were 
not able to gather data from superiors and employees in terms of the C-Sample, made the 
data collection from these stakeholders superfluous. 

With respect to content and process data, we researched the programme descriptions and 
the information available from the business schools. 

The research on the non-customised programmes was done in the period from December 
2000 to May 2001. 



4.2.3.3 Data Analysis 

Having described the way we collected the data, we now take a look at the data analysis 
procedures. The application of STEER produces a variety of data types, which require 
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different methods of analysis. We start with the quantitative methods used for the 
questionnaires as well as for the administrative data. Secondly, we discuss the qualitative 
data from interviews and content analysis. 

Before we go into detail, it is important to note, that all data was made anonymous once we 
had captured it. Each interviewee received an administration number, so that in the analysis 
no names would appear, i.e. the answers were treated in a highly confidential way. 

4.2.3.3.1 Analysis of Quantitative Data 

The majority of the ‘happy sheet’ questions (ca. 80 percent) and ca. half of the telephone 
interview questions consisted of rating questions. For these closed questions we used a five- 
item Likert rating scale (Bortz & Doring 1995) with verbal descriptions for interval ratings 
(from one to five). The reason for using the Likert scale has been its comprehensibility. Likert 
scales are quite common and easy to understand for the interviewees. In particular, we used 
a five scale rating because an important share of the data was collected over the telephone. 
More extensive or complicated scales would have interrupted the flow of the interview and 
would have made it lengthier. The literature regards Likert scales as reliable (Chisnall 1992; 
Bortz & Doring 1995), which was thus another criterion of choice. During design, the 
questions and scales had been tested on their comprehensibility and their selectivity. 

In our decision for the Likert scale, we have acknowledged the discussion in the literature on 
the advantages and disadvantages of five item scales compared to four item scales. A four 
item scale forces the interviewee to decide for a more positive or a more negative 
assessment and therefore eliminates the opportunity to respond neutrally, i.e. choosing the 
middle option. We decided to use a five scale rating because we consider the neutral 
response as an important statement as well. In terms of the evaluation of programmes, we 
understood the middle response as a sign of mediocrity - a statement that carries a 
message to the evaluator as well. Furthermore, in our testing of the questionnaire we did not 
find a significant tendency towards the middle option. 

After the interviews, the responses were coded from 1 (the lowest) to 5 (the highest) and fed 
into an Excel sheet. The interval nature of the responses enabled us to conduct a number of 
statistical analyses for both samples separately. First of all, we determined the 
frequencies and the descriptive statistics including the means and variances of the 
responses. In terms of the comparative study, we were interested in inferential statistics as 
well, which investigate the extent to which the two samples C and NC differ. We thus 
compared the means for significance as a second step. 
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The statistical analysis assisted us in deciding on the null hypothesis which we formulated 
in section 4.2.2. We restate the null hypothesis specifically for the study of the two samples. 

The responses from the two samples originate from the same population and 
therefore there is no difference between the responses/findings of the 
participants from the customised and the non-customised programmes. 

We applied a t-test for independent samples in order to determine whether a significant 
difference exists between the responses of the two samples. However, our interview 
questions included verbal descriptions of interval ratings and we were unsure whether we 
could assume normal distributions for the two samples. Therefore, we applied a non- 
parametric method of analysing the two samples as well. We applied a Mann-Whitney U- 
Test to the two samples with respect to the same null hypothesis. The U-test is commonly 
regarded as a good alternative to the t-test, if one wishes to avoid the assumptions of the 
parametric counterpart (Bortz 1999). Hence, we considered the non-parametric test as 
relevant for our Likert scales. 

For the few questions that provided nominal data, we constructed a non-parametric Chi- 
square Test. The Chi-square Test allows us to determine whether the responses of the two 
samples come from different populations (Siegel 1956). 

Finally, we investigated possible correlations with respect to customisation using a 
Spearman Rank Correlation Coefficient. We prefer the Spearman r over the Pearson r 
because we could not be sure whether our response items on our scales were related in a 
linear fashion (Runyon & Haber 1971; Bortz 1999). 

Apart from the principal focus of the research on customisation, we conducted a variety of 
additional analyses. For example, we studied the effects of the location, the level or the 
business unit background of the executive, as well as the influence by the type of business 
school and the length of the programmes. These analyses have been done by using the 
same techniques as mentioned above. 

We analysed the administrative and controlling data by applying simple statistics like 
percentages, ratios and frequencies. 

4.2.3.3.2 Analysis of Qualitative Data 

The various interviews and questionnaires included open questions that produced qualitative 
responses. The qualitative data has served as a way to understand and interpret the 
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quantitative results. Moreover, it can explain and identify different issues that have not been 
captured by the quantitative analysis. Consequently, we placed special emphasis on the 
analysis of the qualitative results as well. 

In the analysis of the qualitative data we applied the interpretative-reductive content 
analysis by Mayring (1989, 1993 - cf. also Lamnek 1995). According to Mayring’s method 
we performed the following procedures. Firstly, the responses were captured by the 
interviewer through key statements and quotes during the telephone interview. These were 
reviewed directly after the interview in order to complete fragmented phrases from memory 
and to bring the responses into a readable form. These answers then presented the raw 
material to Mayring’s qualitative content analysis. Secondly, the answered were 
paraphrased, i.e. the responses were translated into English (when necessary), ‘cleaned’ of 
non-applicable remarks and classified by question in an Excel sheet. 78 Thirdly, the answers 
were coded or generalised (Mayring 1993), i.e. we reformulated each response in a more 
general manner, so that similar answers would receive the same wording/coding. Finally, we 
were able to reduce the different responses by summing up generalised answers according 
to their codes. As a result, we had a list of generalised responses with their respective 
occurrences (in form of percentages). 

Mayring’s technique enabled us to manage the extensive amount of interview data that we 
gathered. In total, the 106 follow-up telephone interviews included 14 open-ended questions 
each. These amounted to over 1,450 responses of sometimes one-page long texts. 

Moreover, Mayring’s technique resulted in a combination of a verbal statement and a 
percentage, which enabled us to rank the respective responses according to the frequency 
of being mentioned. The technique assisted us in reducing the large number of responses 
into comprehensible results (Bortz & Doring 1995). 

Overall, we would like to point out that we chose a combination of qualitative and 
quantitative techniques. In the corporate framework, numbers and percentages possess a 
greater convincing power. The importance is especially emphasised in a company like 
DaimlerChrysler, which is characterised by a strong engineering culture. Likewise, 
quantitative data enabled us to apply explanatory statistics, which are of interest from an 
academic standpoint. The qualitative data can play a significant role in terms of 
understanding the value creation of customised and non-customised programmes. The 

78 The classification in the Excel sheet allowed fortracing back the major characteristics of the interviewee, i.e. 
the customisation type, the business school, the level or the location for example. 
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analysis provides us with the underlying reasons and details, so that we are able to interpret 
the quantitative results. In the framework of our empirical research we acknowledge that the 
techniques might also have drawbacks and weaknesses. We consider these possible 
limitations in chapter 6. 



4.2.4 Experiences in the Application Process 

The application of STEER to real-life executive education offered several unique research 
opportunities to us. However, at the same time we encountered unforeseen complications 
in the implementation process. In the following section, we would like to review our 
observations that we made during the 24 months in the field. We take a look at our practical 
experiences within the corporate university first. Secondly, we recapitulate our observations 
in the framework of the wider organisation. 



4.2.4.1 Practical Experiences within the Corporate University 

The evaluation project took place in a very dynamic period for DaimlerChrysler as an 
organisation. Our research began only seven months after the merger of Daimler-Benz and 
Chrysler. At that time, most of the organisational changes of the post-merger integration 
process had not yet been cascaded down to the lower levels. Hence, we experienced the 
organisational consequences of the merger for the DCU during our evaluation project. 

The merger implied considerable personnel changes for the DCU as a global institution 
within DaimlerChrysler. After the merger, the DCU was staffed from both halves of the 
company. Chrysler had not possessed a corporate university before the merger, but had its 
own expertise and competencies in the field of management development. The respective 
employees and managers joined the staff of the Daimler-Benz Corporate University to form 
the DaimlerChrysler Corporate University. These organisational changes had the following 
consequences for application of STEER. 

The focus on the evaluation project changed. At the start of the research all programmes of 
the corporate university had just been launched. Consequently, the need for evaluation was 
immanent, driven by the common questions of “How did we do?” and “What have we 
achieved?”. At that time, there was a common awareness towards the evaluation project. 
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Moreover, there existed a direct supervision and support by the management of the 
corporate university. 

The merger then implied that most programmes had to be re-launched and redesigned to 
suit the needs of the new company. Additionally, new staff had to be integrated into the DCU. 
These events implied a change in priorities for the DCU staff. The focus on evaluation 
decreased substantially during the period of change . 79 Therefore, fewer organisational 
support as planned was possible from the management of the DCU with respect to our 
evaluation. Hence, we were unable to realise the application of STEER with the same speed 
and extent as originally devised. During the period of change, we had to reduce some of the 
evaluation activities, for example, we refrained from conducting several stakeholder 
interviews since we did not have the necessary access and resources . 80 

These changes represented an additional challenge to our research. The overall plans for 
the evaluation with STEER had to be reconsidered and re-focused to a different scale. 
However, we were careful not to change the overall nature of the research. The underlying 
rationale and the quality of results remained largely unaffected. 

In contrast to the organisational challenges, we experienced a general openness to 
evaluation. All members of the DCU saw the need for evaluating the programmes. We found 
that the evaluation procedure was not only viewed as a way to state one’s achievements 
(summative evaluation), but also to see it as an opportunity to improve (formative 
evaluation). The application of STEER to the DCU programmes can be compared to a 
‘green-field’ evaluation, i.e. little practices existed beforehand to assess the impact of 
programmes. At the start of the research project the only evaluation efforts were led by the 
business schools from the outside . 81 We found that the application of STEER had a 
significant impact in terms of information available on the effects of the programmes. Our 
evaluation provided data that did not exist beforehand and thus delivered new insights on the 
DCU programmes. 

However, we were also able to discern the political significance of an evaluation. The 
application of STEER implied that the outcome of the DCU activities became more 
transparent. Thereby, the application of STEER established new ways of judging the 

79 The interest in evaluation rose again right at the end of the time in the field, when most post-merger challenges 
had been mastered. 

80 The access to important stakeholders depended on the network and the reputation of an important person of 
the DCU, for example a Senior Manager. He/she established the first contact and explained the need for the 
interview. ‘On our own’ we experienced difficulties to access important stakeholders. 
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performance of the executive programmes. We realised that the evaluation data had the 
potential to illustrate the strengths, but also the weaknesses of programmes. The results of 
the evaluation enabled direct inferences on the work performance of staff members. Our 
work possessed a strong political relevance for the stakeholders within the DCU and thus 
placed considerable responsibility in our hands. 

Overall, we learned from our experience that a research period of two years proves to be a 
long time in the dynamic pace of today’s business environment. The people involved in the 
research and the research setting itself cannot be assumed to remain stable in a real world 
setting like in our case. We consider it more appropriate to expect constant changes in an 
evaluation project, which should be accounted for by the necessary flexibility of the 
evaluation tool. 



4.2.4.2 Organisational Circumstances within DaimlerChrysler 

The large size of DaimlerChrysler - comprising ca. 7,000 executives - enabled us to conduct 
the research design as described above. At the same time the nature of the organisation 
rendered the evaluation research more complex. The organisational circumstances and the 
sheer size of DaimlerChrysler affected the application of STEER on the following aspects. 

Firstly, we experienced that the data streams in such a large and fast changing organisation 
become very complex to handle. DaimlerChrysler experienced important global changes in 
terms of its structure and its scope of business. Certain pieces of information were thus not 
available for the entire target group at the time of the evaluation. For example, as a result of 
the merger the two different executive appraisal systems of Daimler-Benz and Chrysler were 
merged into a new one. This implied that new measures and indicators were devised. The 
appraisals for a given executive became discontinuous with respect to the format and the 
content of his/her personal file. We were thus unable to use executive ratings in the 
evaluation by STEER. Most of the programmes that we studied took place between 1997 
and 2000. Unfortunately, the new appraisal systems was changed in 1999/2000. Therefore, 
we were unable to compare possible changes in the ratings of executives after the 
programme with the ratings before the programmes due to the change in the system. 



81 The three business schools conducted end-of-programme evaluation. 



176 




Secondly, within DaimlerChrysler important business units are structured in a decentralised 
way. These business units therefore possess their own distinctive personnel systems 
including own training and development institutions. Consequently, it has been difficult to 
distinguish the influence of the DCU programmes from other training and development 
interventions within the company. Many executives have visited more than one programme 
in a given year since the DCU does not have the ‘monopoly’ on corporate education. Given 
the very complex causality structures of the effects of executive education, the evaluation at 
DaimlerChrysler encountered additional inferences by other training and development 
activities. For example, no suitable control group could be found within the Total Population. 
Hence, we decided to conduct a comparative study as a reference point, i.e. a quasi-control 
group. The comparative study was possible on the grounds of the decentralised training and 
development practices, which allowed executives to visit management development 
programmes on their own initiative. 

Thirdly, the organisation has been subject to incremental change, especially in the 
aftermath of the merger. The constant change has made it difficult to trace all the important 
stakeholders of the DCU management development programmes. The executives of the 
target group moved to new assignments and the important sponsors changed. For example, 
in terms of participants we learned that the executives change their job every four to five 
years on average. This implies that in any given year 20 percent of the 459 former 
participants 82 move within the company globally, which amounts to ca. 90 executives a year. 
These changes must be captured in order to be able to trace the executives. Furthermore, 
important sponsors change their jobs or leave the company. For example, four of the five 
most important initiators of the corporate university left the company during the evaluation 
project. Considering the time horizon of the evaluation study, these changes imply that 
longitudinal research becomes very complicated to conduct. It proved the right decision to 
conduct a comparative study at one point in time. 

Finally, the size of the organisation slows down the overall impact of the executive 
programmes. The DCU is unable to teach all 7,000 executives within one year. Therefore, it 
takes several years for a management development programme to reach all members in its 
target group. As we have seen, the 418 participants of the C-Sample represent not even 10 
percent of the total executive population of DaimlerChrysler. As a result, many of the DCU 
programmes have not yet reached a ‘critical mass’ in terms of their target group. The slow 
impact of the executive education makes it more difficult to discern any effects in terms of 
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broader organisational measures. In the case of a smaller organisation, all managers could 
be trained within a short period of time. The effects of the programme would then have a 
more concise impact on the organisation as a whole. In the case of DaimlerChrysler, the size 
of the organisation ‘dilutes’ the impact of the executive programmes of the DCU on the 
grounds of its limited capacities and thus makes the results of the programmes more difficult 
to identify. 

Overall, we have no way of putting our experiences into perspective and compare them to 
the evaluation research in other large industrial multi-nationals. Thus, it is difficult to judge 
the nature of the evaluation experience within DaimlerChrysler. However, we can infer from 
our experiences that the size of an organisation is likely to influence the evaluation research 
by making it more complex. 

Concluding chapter 4 - we described the field of research and outlined the application 
process of STEER. We elaborated on the impact of organisational changes in the evaluation 
process and explained that consequently some evaluation instruments could not be used to 
their full extent. However, we focused on the instruments at hand and applied STEER to a 
sample of customised programmes and to a sample of non-customised programmes. The 
results of the application of STEER can be found in the next chapter. 



82 The figure 459 consists of 418 participants of the C-Sample and 41 participants from the NC-Sample. 
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5 Results 



" Assessing the practical benefit of the programme is very difficult to do." 

(Participant from the NC-Sample) 

In the framework of our thesis we achieve two purposes with the application of STEER. One 
the one hand, we can assess the quality of the tools itself. In our thesis we combine the 
theoretical development of a new tool with a practical testing of it. As a result, we are able to 
provide a first critical assessment of STEER, which assist us in demonstrating the strengths 
and weaknesses of the tool. On the other hand, we are able to research the value creation 
of executive management development programmes by comparing customised and non- 
customised programmes. This has been the additional, specific purpose of our research. 

Therefore, we structure the following chapter on the results of the research into two 
dimensions. Dimension one comprises the results provided by STEER, which helps us in 
answering our research questions on customisation (CS1 - CS5). Dimension two takes a 
step back and looks at the application and the quality of the results of STEER in the wider 
framework of evaluation research. 



5.1 Dimension One: Results of the Comparative Study 

“Customisation, dear to the heart of corporate training, is the key to a successful 
partnership between corporate universities and business schools. ” 

(Smith 1997, p. 1) 

The application of STEER to the field has generated results with respect to the four 
perspectives. We take a look at the characteristics of the two samples first in order to 
understand the scope of the evaluation. Then, we present the results, which the four 
perspectives of STEER have generated. Finally, we interpret the findings and discuss the 
repercussions for the field of research. 



5.1.1 Description of the C-Sample in Comparison to the NC-Sample 

In the following section we distinguish between programmes’ characteristics, i.e. describing 
the kind of programmes that the participants have visited, and participants’ characteristics, 
i.e. providing information on the executives in both samples. 
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5.1. 1.1 Programmes’ Characteristics 



First of all, we take a look at the business schools in the two samples. 




Figure 5.1 : Represented Business Schools 

Source: STEER - Administrative Data from DCU and Business Schools 

Figure 5.1 demonstrates that the two samples are similar to each other in terms of the share 
of business schools. All three business schools are known for their high-profile professors 
and their high quality executive education. Furthermore, all three deliver customised 
programmes to the DCU. 

In the framework of the three business schools we included the following programmes in our 
two samples. The DCU offers four customised programmes in co-operation with the business 
schools, which are for internal executives only. The 22 non-customised programmes are 
offered by the same business schools on the open market. Executives from different firms 
and backgrounds can apply to and take part in these non-customised programmes. 



Business Schools 


Customised Programmes 


Non-Customised Programmes 


Harvard Business School 
(Boston, USA) 


Managing Global Opportunities 


Driving Corporate Performance 
Creating Corporate Advantage 
Advance Management Program 
The General Management Program 
Program Global Leadership 

Strategic Human Resource 
Management 



(continued on the next page) 
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Business Schools 


Customised Programmes 


Non-Customised Programmes 


INSEAD 

(Fontainebleau, France) 


Managing Business 


Managerial Skills for International 
Business 

Advanced Industrial Marketing Strategy 
Advanced Management Programme 

Hewlett Packard Management 
Academy 

International Executive Programme 

International Manufacturing 
Programme 

Strategy in Eastern Europe 
Strategic R&D Management 
Young Managers Programme 


IMD 

(Lausanne, Switzerland) 


Driving Success in a Global Company 

Leadership in a Cross-cultural 
Environment 


International Program for Board 
Members 

Program for Executive Development 
Orchestrating Winning Performance 
Managing Corporate Resources 
Mobilising People 
Managing for Marketing Success 
HR Leadership in Global Organisations 



Table 5.1 : Overview of Programmes Included in the Two Samples 

Source: Administrative Data from DCU and Business Schools 



The non-customised programmes are similar to the customised programmes in several 
aspects. Firstly, they are run by the same business schools. This is important because we 
can assume that the participants have experienced a similar learning environment. The 
programmes of the two samples are taught, to a large extent, by the same professors . 83 

Secondly, all programmes rely predominantly on the case study method and therefore their 
underlying teaching methodologies are very similar. The case study method reviews a 
particular real-life situation and extracts the learning from the discussion of it. It is a 
practically oriented form of teaching. Hereby, the common procedure is to prepare the case 
study in small groups in order to discuss it in the plenum. 



83 Even if the programmes are taught by different professors, we can still infer a similar quality of professors. The 
respective business schools belong to the best in the world and guarantee a certain standard of teaching. 
Therefore, we feel it is save to assume, that the programmes in the two samples are run by a similar type and 
quality of professors. 
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Thirdly, the programmes address the same target group. Most of the participants stem from 
the higher middle management according to our terminology in chapter 2. We therefore 
selected the sample accordingly (as to be seen in figure 5.4). 

Finally, and this a very important part, all programmes are dealing with strategy or strategic 
thinking in one form or another, i.e. strategy process capability in our terminology. This can 
be seen in the course descriptions, which are presented in appendix 8.11. In most 
programmes, the section on strategy is embedded in a wider curriculum of general 
management skills. Only one programme focuses solely on making and implementing 
strategy - Harvard’s Creating Corporate Advantage. 

However, it is clear that the two samples C and NC cannot be identical in terms of the 
programmes included. There are several criteria on which the programmes differ. First of all, 
the specific orientation of the programmes varies within the given boundary of general 
management and leadership skills. We consider this as acceptable because they all include 
strategy and strategic thinking. 

Secondly, the programmes differ in terms of participant group sizes. Some of these 
programmes (internal as well as external) include 30 participants, whereas other include up 
to 60 participants. These numbers can vary according to demand. We can accept these 
differences because the overall nature of the programmes remains similar. The overall size is 
balanced by the fact that most programmes use small work groups so that a lot of interaction 
takes place in groups of five to ten people. 

Finally, the lengths of the programmes differs between the two samples. Figure 5.2 (on the 
next page) illustrates that some of the non-customised programmes are substantially longer 
than the customised programmes. 84 We take the difference in length seriously because we 
assume that a longer programme has a more intensive effect on the participant with respect 
to a number of criteria - as for example the individual learning or the networking effect. 
However, we assume also that some evaluation criteria are affected very little by the length 
of the programme, as for example the relevance of the content to DaimlerChrysler. 
Therefore, we decided to consider all responses at first in order to achieve a larger ‘n’. For 
questions, which we consider to be highly influenced by the length of the programme, we 



84 We acknowledge that the customised programmes differ in lengths themselves. However, the two weeks 
programmes are taught in two one-week modules, i.e. provide a similar programme experience to the one week 
programmes. Furthermore, we assume that the difference between one and two weeks is not so substantial that 
it changes the nature of the programmes significantly. 
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then ‘trim’ the NC-Sample as seen in figure 5.2. In the analysis, the results for the trimmed 
NC-Sample can be found in a line below the respective figure. 



C-SAMPLE 

Lengths of Programmes in Days 




NC-SAMPLE 




Figure 5.2: Lengths of Programmes in Days 

Source: STEER - Administrative Data from DCU and Business Schools 

To conclude this subsection - we have selected two samples of programmes, which are 
different with respect to customisation. However, in terms of their content, strategy is an 
important element in all programmes regardless their degree of customisation. Hence, we 
can investigate the effects of customisation since the two programmes in the two samples 
are similar in their setting and structure. The programmes represent the same business 
schools, the same target group (level), similar professors and the same teaching 
methodology. As a compromise, we accept differences in the group sizes and the 
programmes’ lengths - which we account for in our analysis. 
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5.1. 1.2 Participants’ Characteristics 

In the following subsection we consider the characteristics of the participants of the two 
samples. We take a look at the different management levels of participants in the samples, 
the share of male to female participants, their business unit backgrounds, their locations and 
the time passed since they followed the programmes. 

To start with, figure 5.3 shows that the two samples are similar in terms of the three 
DaimlerChrysler management levels represented. The emphasis of the study has been on 
Directors (cf. section 4.1.3) because they form the heart of the target group of the DCU 
programmes and the external programmes. 




Figure 5.3: Executive Levels of Participants 

Source: STEER - Administrative Data from DCU and Business Schools 

Table 5.2 tells us the share of male to female participants. We can see that both samples 
consist of predominantly male participants. These shares represent the overall distribution of 
male to female executives within DaimlerChrysler. 



Sample 


Male 


Female 


C-Sample 


89% 


11 % 


NC-Sample 


91 % 


9% 



Table 5.2: Share of Male to Female Participants 

Source: STEER - Administrative Data from DCU and Business Schools 



Even the open programmes of the selected business schools comprise predominately male 
participants. A recent survey of the Financial Times showed that women represent the 
minority in non-customised programmes (the survey lists proportion of female participants: 
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Harvard 34%, INSEAD 28% and IMD 22%; Bradshaw 2001, Financial Times Executive 
Education Survey 2001). 

In figure 5.4 we illustrate the distribution of the business unit backgrounds of participants. 



Business Units of Participants in the C-SAMPLE 




Units 



Business Units of Participants in the NC-SAMPLE 




Units 



Figure 5.4: Business Units of Participants 

Source: STEER - Administrative Data from DCU and Business Schools 



The high share of Other Business Units in the NC-Sample is striking. The difference might be 
caused by a number of factors. One of the reasons might be that the Other Business Units 
are often smaller subsidiary companies, who do not have their own internal training 
department. Therefore, these smaller units send their executives more often to external 
programmes than the larger business units (who do have own training departments). 
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Furthermore, we split up the participants in the two samples according to their location in 
order to show the global nature of the programmes - as depicted in figure 5.5. 





Figure 5.5: Locations of Participants 

Source: STEER - Administrative Data from DCU and Business Schools 

A large number of participants of both samples come from Germany - 62 percent in the case 
of the C-Sample and 47 percent of the NC-Sample. Furthermore, the dominance of the 
Stuttgart region becomes apparent, almost one-third of the C-Sample and ca. one-fifth of the 
NC-Sample come from the home region of the former Daimler-Benz AG. The USA is 
represented by similar shares in both samples. The differences with respect to the Rest of 
Europe and Rest of World are striking. Generally, it implies that the non-customised 
programmes attract more participants from Europe, but include fewer participants from the 
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Rest of the World. We assume that those Rest of the World participants utilise their local 
business schools and thus do not travel to Europe for executive education. Whereas 
participants of the internal, corporate programmes travel to Europe to take part in corporate 
executives education. 

Figure 5.6 shows a difference of the two samples. It represents the time passed since 
programme - measured at the interview dates (early 2000 and early 2001). The relatively 
recent programme dates of the C-Sample depend on the short existence of the DCU (start of 
operation in 1998). 



Time Passed Since the Programmes for the C-SAMPLE 

100% 

75% 

50% 

25% 

0% 

6 Months 12 Months 24 Months 36 Months 48 Months 60 Months 




Time Passed Since the Programmes for the NC-SAMPLE 




Figure 5.6: Time Passed Since the Programmes 

Source: STEER - Administrative Data from DCU and Business Schools 
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The participation of DaimlerChrysIer executives in non-customised programmes stretches 
back over more than 20 years. We decided to include only those participants in the NC- 
Sample who had been to an open programme within the last five years. 

We examined whether the differences in the time passed since the programmes has caused 
any distortion for our results, but we did not find any significant influences. To test this, we 
consolidated the two samples into one total batch. Then we grouped the total batch into 
recent participation (6 to 12 months, n=77) and antecedent participation (24 to 60 months, 
n=29). Subsequently, we analysed the differences with respect to the answers to several 
questions - as for example ' enjoyment of the programme ’ and the ‘personal value of the 
programme’. We found that the two groups did not differ significantly 85 and that there was no 
visible effect on the grounds of the time passed since the programme. 

Finally, we studied the objectives for the participation in the programmes. Figure 5.7 (on the 
next page) outlines the results of the inquiry into the initiation of the programme. We 
discern an important difference in terms of the person who induced the executive to take 
part. 

In the case of the C-Sample most of the participants were nominated by the central human 
resource department EMD. 86 This implies that executives are selected to participate (50 
percent of all cases in the C-Sample). Only about a fourth of participants took an active role 
and applied themselves. These results reflect the fact that most DCU programmes are 
nomination based and only few places are allocated through direct application. 

In contrast, most participants of the NC-Sample took part in the programmes out of their own 
initiative. These executives took a more active role and organised the programme 
themselves. Paauwe and Williams (2001) specified the differences between the two 
programme types. Customised programmes are “pushed” by the centre, whereas non- 
customised programmes are more “pulled” by the participants. 

Furthermore, we can see that the superior played less dominant role in both samples. 



85 The numbers for enjoying the programmes are - 4.51 for the recent and 4.63 for the antecedent participation. 
The personal values compares in the following - 4. 46 for the recent and 4.24 for the antecedent participation. 

86 Executive Management Development is the corporate function within DaimlerChrysIer that supports the 
executives in their personal and professional (career) development. EMD usually decides whether an executives 
can attend a programme. 



188 




C-SAMPLE 



Who initiated the idea of your participation in the 
programme? 

100% 

75% 

50% 

25% 

0% 



50% 




Nomination by Idea of Own Initiative Other NO ANSWER 
EMD Superior 



NC-SAMPLE 

Who initiated the idea of your participation in the 
programme? 




| Trimmed (22%) (26%) (48%) (4%) (0%) 

Figure 5.7: The Initiation of Programmes 

Source: STEER - Follow-up Telephone Interviews 

In table 5.3 (on the next page) we list the qualitative responses (analysed with interpretative- 
reductive content analysis of Mayring, 1989, cf. subsection 4. 2.3.3. 2) for the objectives of 
the participation. 



These results reflect the general application procedure for the programmes of the two 
samples. The customised DCU programmes are administrated centrally and the majority of 
participants are selected for participation. The participation in the non-customised 
programmes arises predominantly from an own initiative of the executives. 
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C-Sample 


n=72 


Ranking 


“What were the objectives for your participation in the programme?” 


Replies 


Percentage 


1 


Being selected by EMD 


34 


47% 


2 


Seeking new business and professional skills 


15 


21% 


3 


Being interested by the content description of the programme 


13 


18% 


4 


Building general management skills 


13 


18% 


5 


Arising from my personal interest 


12 


17% 


6 


Networking with internal executives 


12 


17% 


7 


Being initiated by the superior 


10 


14% 


8 


Building a more global understanding of DaimlerChrysler 


9 


13% 


9 


Developing myself further generally 


9 


13% 


10 


Preparing me for a new or future assignment 


9 


13% 


11 


Being attracted by the name of the business school 


8 


11% 


12 


Enhancing strategy skills and knowledge 


7 


10% 


13 


Exchanging personal and professional experiences 


7 


10% 


14 


Wanting a global business orientation 


7 


10% 


15 


Learning inter-culturally 


6 


8% 



NC-Sample 


n=34 


Ranking 


“What were the objectives for your participation in the programme?” 


Replies 


Percentage 


1 


Preparing me for a new or future assignment 


17 


50% 


2 


Seeking specific professional skills in business 


11 


32% 


3 


Arising from my personal interest 


9 


26% 


4 


Being initiated by the superior 


8 


24% 


5 


Refreshing existing business skills and knowledge 


8 


24% 


6 


Being part of a wider HRD scheme 


6 


18% 


7 


Being rewarded for my performance 


6 


18% 


8 


Amending my professional skills and knowledge 


5 


15% 


9 


Being selected by EMD 


5 


15% 


10 


Seeking inspiration and new ideas 


5 


15% 


11 


Wanting a global business orientation 


5 


15% 


12 


Enhancing strategy skills and knowledge 


4 


12% 


13 


Meeting and exchanging with external executives from other companies 


4 


12% 


14 


Being attracted by the name of the business school 


3 


9% 


15 


Learning from other companies and industries 


3 


9% 



Table 5.3: The Objectives for the Participation in the Programme 

Source: STEER - Follow-up Telephone Interviews 



The most common objective seems the desire to ‘build, refresh and develop new skills’ - a 
very practical orientation. These skills concern foremost comprehensive business and 
management skills, out of which strategy process capability is mentioned separately by a 
share of 10 to 15 percent (similar in both samples). 
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In the case of the C-Sample some interesting results are the focus on ‘networking’ (17 
percent) and on the motivation to build a ‘better understanding of DaimlerChrysler’ (13 
percent). Furthermore, the ‘status’ of visiting the respective business schools seems 
important for participants (1 1 percent). 

In the NC-Sample the participation seems often directly linked to a new management 
assignment - half of the NC-Sample stated that the programme was meant as a ‘preparation 
for a new job’ (C-Sample only 13 percent). Furthermore, the programme was seen as a 
‘reward’ for some participants (18 percent). In a number of cases the executives took part in 
the open programme on purpose in order to learn from other companies (nine percent). 

Both samples illustrate very specific objectives as well. Three participants from the C-Sample 
said that they took part in order to test the new offer by the DCU for their HR department in 
their respective business units. Several executives emphasised the fact that the programme 
was in English and that it thus represented a good language training for them. Many 
participants stated that the programme was meant to provide new ideas and inspirations and 
to broaden their international horizons. 

To summarise - the characteristics of the participants in the two samples are similar with 
respect to their personal and business backgrounds. However, the participation has often 
been initiated differently (nomination vs. self-interest). The customised programmes are 
predominately staffed via a nomination process. The open, non-customised programmes 
arise often from a personal motivation. The objectives for going have been partly similar - 
seeking new skills generally - but also quite different with respect to customisation. 



5.1 .2 Results of the Four Perspectives of STEER 

In the following section we present the results of the evaluation by STEER. The evaluation 
approach enables us to compare the results for the C-Sample and the NC-Sample for each 
perspective. The section is therefore structured according to the four perspectives 
stakeholder, content, process and business 87 



87 The conceptional basis for the following results have been described in chapter 3, part 3.3. The application of 
the tool to the field has been illustrated in chapter 4, part 4.2. 
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5.1. 2.1 Results from the Stakeholder Perspective 



“A programme, which is tailored directly for the job does not exist - so the 
programme helps, but participating in the programme is not a precondition for 
success [in the job]. ” 



(Participant from the NC-Sample) 

We have structured the results of the stakeholder perspective into quantitative and qualitative 
indicators (cf. subsection 3.3.2. 1.4), which we deal with in turn. 

5.1. 2.1.1 Quantitative Indicators 

In terms of the quantitative indicators we have suggested to evaluate the overall value rating 
of participants, the benchmark to other programmes as seen by participants, call-back and 
cancellation rates by participants and attendance rates of sponsors at programmes. We now 
take a look at each of these indicators. 

Figure 5.8 (on the next page) outlines the overall value rating of participants. On the 
whole, the value ratings of the C-Sample and the NC-Sample are very similar in terms of 
their rating averages and their distributions. The participants of both programme types seem 
equally content with respect to the personal value derived from the programme. 

Respecting the difference in programme lengths, we can discern a slight difference in the 
comparison of the C-Sample and the trimmed NC-Sample. Trimming the NC-Sample, the 
distribution changes and the average of the trimmed NC-Sample is 0.21 points lower than 
the C-Sample. The difference implies that customised programmes receive slightly better 
value ratings than the non-customised ones. 

However, applying a Mann-Whitney U-Test we find a significance level of only 15 percent 
(significance level a C toNc = 0.154). So we cannot be sure whether the difference marks an 
significant distinction between the C-Sample and the trimmed NC-Sample. 
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C-SAMPLE 



Please rate the value of the programme for you 
personnally. 



100% 

75% 

50% 

25% 

0% 



n = 72 

0 4.43 

50% 




Very High High Medium Low No Value NO 

ANSWER 



NC-SAMPLE 

Please rate the value of the programme for you 
personnally. 



100% n 


1 










0 4.32 


75% - 












(Trimmed 0 4.22) 


50% - 


■ 


38% 










25% - 


■ 


■ 


15% 








0% - 


u . 


■ 




0% 


0% 


0% 


Very High 


High 


Medium 


Low 


No Value NO 














ANSWER 


Trimmed 


( 35 %) 


( 52 %) 


( 13 %) 


(0%) 


(0%) 


(0%) 



Figure 5.8: Overall Value Rating of Participants 

Source: STEER - Follow-up Telephone Interviews 

Generally, we observe very positive value ratings for both programme types. Ratings of 4.0 
and higher demonstrate a perceived high value in follow-up surveys (cf. Liedtka et al. 1999). 

The next quantitative indicator introduces the benchmark to other programmes as seen by 
participants. 

In figure 5.9 (on the next page) we can see that the ratings are again similar. The distribution 
and the average of the NC-Sample lies slightly above the C-Sample. However, the C-Sample 
and the trimmed NC-Sample seem almost identical. Furthermore, the statistical analysis 
does not reveal any significant differences. 
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C-SAMPLE 



100% 



How did the programme compare to similar management 
development programmes that you have participated in? 

n = 72 

1 0 4.51 1 



75% A 



51% 




Oneofthe Betterthan Average Worse than Oneofthe NO 
best average average worst ANSWER 



NC-SAMPLE 



How did the programme compare to similar management 
development programmes that you have participated in? 



n = 34 

4 nn o/ 


1 uu /□ - 

75% - 


59% 










0 4.63 
(Trimmed 0 4.52) 


50% - 


■ 


26% 










25% - 
0% - 


■ 




3% 


0% 


0% 


12% 


One of the 


Betterthan 


Average 


Worse than 


Oneofthe NO 




best 


average 




average 


worst 


ANSWER 


Trimmed 


(52%) 


(35%) 


(4%) 


(0%) 


(0%) 


(9%) 



Figure 5.9 : Benchmark to Other Programmes as Seen by Participants 

Source: STEER - Follow-up Telephone Interviews 

The results show that both types (C and NC) are regarded as top-range, premium 
programmes within the area of executive education. In over 50 percent of all cases, the 
executives stated that the programme has been one of the best that they have visited. 

The next set of indicators outlines the call-back and cancellation rates by participants in 
table 5.4 (on the next page). In contrast to the indicators above, these rates rely on 
administrative data and not on participants’ perceptions. 
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Sample 


Call-back Rate 


Cancellation Rate 


C-Sample 


1.2% 


14.8 % 


NC-Sample 


7.3 % 


10 % 88 



Table 5.4: Call-back and Cancellation Rates by Participants 

Source: STEER - Administrative Data from DCU and Business Schools 



The call-back rate suggests that the non-customised programmes have a higher rate of 
participants returning to the business school for another programme (7.3 percent) compared 
to the customised programmes (1.2 percent). However, we must be careful with these 
figures. Not all customised programmes have an open enrolment application procedure like 
the non-customised programmes. Some programmes in the C-Sample rely on a nomination 
system that limits the number of call-backs. 

The cancellation rate possesses a greater meaningfulness. Cancellations occur usually when 
participants, who have already registered, cannot attend the programme because ‘something 
more important’ takes place at the same time as the programme. The cancellation rate can 
thereby be regarded as an indicator for the perceived importance (i.e. priority) of the 
programme by its participants. 89 If an executive development programme is regarded 
commonly as a unique opportunity and an honour, then few executives will dare to cancel 
their participation. For example, the Work-Out programme with Jack Welch 90 at General 
Electric’s corporate university at Crotonville has a very low cancellation rate (Stewart 1991). 
Cancellations for the Work-Out programme must be well justified in front of Mr. Welch 
himself. 

The cancellation rates in table 5.4 portray an interesting picture. The internal cancellation 
rate of the customised programmes is higher (14.8 percent) than the external rate of the 
non-customised programmes (ten percent). 

One way of interpreting this finding is that participants have less inhibitions to cancel the 
customised programmes because they are internally run. For example at DCU, participants 
of the customised executive programmes do not have to bear the cancellation fee. Another 
explanation could be that the internal programmes have not yet attained the status of 

88 The cancellation rate for the external, non-customised programmes was not measured by us, but resulted from 
interviews with the controlling staff at the respective business schools. 

89 The underlying logic of the cancellation rate can be seen in the following example. It is quite likely that a 
manager will cancel a scheduled presentation to his/her staff in the case that he/she receives a sudden invitation 
to present to the board of management. However, it is highly unlikely that the manager will cancel a scheduled 
presentation to the board of management due to an unexpected opportunity to present to his/her staff. 

90 Jack Welch is the CEO of General Electric. 
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importance within the organisation. Executives would rank them lower in importance than 
other business duties. Most of the external non-customised programmes are self-initiated. 
Therefore, the cancellation rate might be lower as the participants take a higher personal 
interest in the programme. We suggest further research to identify the reasons for the higher 
cancellation rate of customised programmes. 

DCU could achieve a lower cancellation rate, if it were stricter on cancellations, i.e. passing 
on the cancellation fee to those executives that have cancelled on short notice or ‘ban’ them 
from executive education for a certain period of time (e.g. two years). We are certain, that if 
the organisational pressure was more discernible, then executives would dare less to cancel 
on short notice. 

Finally, we take a look at the attendance rates of sponsors at programmes. The 
comparison to non-customised programmes is problematic with respect to attendance rates. 
Sponsors usually speak only at customised programmes due to their internal nature. A 
presentation given by a sponsor is very rare at non-customised programmes. The results of 
table 5.5 reflect this state of the world. The comparison would be more relevant in the case of 
evaluating two internal programmes through STEER. 



Sample 


Programmes with Sponsor 
Attendance 


Attendance Rates of Sponsors 


C-Sample (14 programmes) 


5 


35.7% 


NC-Sample (29 programmes) 91 


0 


0% 



Table 5.5: Attendance Rates of Sponsors at Programmes 

Source: STEER - Administrative Data from DCU and Business Schools 



Therefore, customised programmes have the advantage of sponsors attending the 
programmes. Internal speakers, e.g. member of the top management, can thereby mediate 
important information on the strategic development of the company and can thus build 
strategy knowledge (cf. subsection 2. 2. 1.1). Furthermore, a speech from a board member 
can foster the identification and motivation of participants. These opportunities are not given 
in the open, non-customised programmes. Overall, we consider the direct contact with high 
level managers as important for the impact of these internal executive programmes. Some 
responses from former participants reflect these. 



91 The NC-Population includes 41 participants, but in several cases two executives have participated in the same 
programme. 
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5.1. 2.1. 2 Qualitative Measures 



We suggested three qualitative measures - fulfilment of participants’ expectations , degree of 
satisfaction of sponsors 92 and changes in the performance ratings as assessed by an internal 
performance appraisal system. 



The fulfilment of participants’ expectation refers to the objectives of the programme as 
stated in table 5.4 (in subsection 5. 1.2. 1.1). Below, table 5.6 lists the replies concerning the 
fulfilment according to frequency. 93 



C-Sample 


n=34 


Ranking 


“To what extent were these objectives fulfilled?” 


Replies 


Percentage 


1 


Yes, completely 


18 


53% 


2 


Met various participants from the different business units, which was valuable 


6 


18% 


3 


Received many new impulses and ideas 


4 


12% 


4 


My expectations were met to a large extent 


3 


9% 


5 


Partly - yes and no 


3 


9% 




Refreshed my skills and knowledge 


3 


9% 



NC-Sample 


n=34 


Ranking 


“To what extent were these objectives fulfilled?” 


Replies 


Percentage 


1 


Yes, completely 


17 


50% 


2 


Received many new impulses and ideas 


6 


18% 


3 


Met various participants from different companies, which was valuable 


6 


18% 


4 


My expectations were met to a large extent 


4 


12% 


5 


The content was not completely new 


4 


12% 




Partly - yes and no 


3 


9% 




Wrong programme - was not relevant for me/did not suit my needs 


3 


9% 



Table 5.6: Fulfilment of Participants’ Expectations 

Source: STEER - Follow-up Telephone Interviews 



In both samples, the majority of participants stated that the programme fulfilled their 
expectations (C-Sample 53 percent, NC-Sample 50 percent). Again, the evaluation reveals a 
similar high quality assessment by the participants of both groups. The results above 
indicate as well the importance of meeting participants from other business units (or other 
companies) and of receiving new impulses and ideas. However, the results of the two 
samples also show room for improvement. Adding the responses ‘partly - yes and no’ and 



As discussed in section 4.2. 3. 2 we were unable to collect sufficient data from superiors and subordinates. 
93 The size of the C-Sample is 34 former participants. This question was added to the second batch of 
customised telephone interviews (cf. subsection 4.2.3.2) and therefore was posed to half of the C-Sample. 
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‘my expectations were met to a large extent’ we can see that some participants were not 
completely satisfied (C-Sample 18 percent, NC-Sample 21 percent). 

An interesting result is the response ‘wrong programme - was not relevant forme/did not suit 
my needs’. In some cases (ca. ten percent of the NC-Sample) participants have taken part in 
a programme, which proved inappropriate for them. For example, one participant stated the 
following. 

“The recommendation of my superior [to participate in the programme] was 
wrong - but this is difficult to communicate back to him. ” 

(Participant from the NC-Sample) 

The quote shows a possible drawback of non-customised programmes. If an external 
programme is self-initiated, then there is a greater danger of choosing an inappropriate form 
of executive education. Customised programmes are run and offered by internal professional 
HRD staff who can advise participants more effectively on executive education. 

The responses by participants show the variation in fulfilled expectations. Some executives 
show their appreciation for the possibility to participate. 

“INSEAD, that was really worth while. INSEAD was a dream of mine and that 
dream came true for me. ” 



(Participant from the C-Sample) 

Here, the aspect of ‘rewarding’ the executive becomes especially apparent. The quote 
demonstrates the prestige that the participation might carry. We can see that companies are 
able to use the executive programmes also for non-learning purposes as for example 
demonstrating special recognition for the participant or rewarding the executive. However, 
not all participants consider the participation as a reward. A participant of the NC-Sample 
shows the opposite scenario. 

“Although, in my programme, I had the feeling that there were a lot of people, 
where you could feel that their companies had sent them to participate, so they 
were not very motivated. ” 



(Participant from the NC-Sample) 
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The aspects of reward and compulsion can be found in the responses of both programmes 
types. Thereby, it seems more likely that participants are more motivated when they have 
initiated the participation in the programme themselves (cf. figure 5.7). 

The results of the degree of satisfaction of sponsors rely on some few qualitative 
interviews. The responses can provide some opinions on the differences of customised to 
non-customised programmes. 

In terms of the value provided, most sponsors consider customisation as the key value 
driver. In the view of sponsors, conducting the executive education programmes internally, 
provides the opportunity, but also the obligation to customise the content. 

“One of the measures of the training and development programmes is to support 
the realisation and implementation of corporate strategy. Thereby, the 
programme serves to communicate the strategy to the executives and at the 
same time to qualify the executives, so that they can implement a given strategy. 

... This customisation cannot be accomplished by an external business school.” 

(Sponsor interview) 

In the above quote we can see the expected role of executive programmes to build strategy 
process capability. The following quote emphasises the importance of strategy knowledge, 
which is mediated by the customised programmes. 

7 see the corporate university in a supporting role with respect to building an 
understanding for the strategic position [among our executives]. ” 

(Sponsor interview) 

In general, most sponsors discern a clear need for customisation of content. However, some 
are also critical with respect to the extent that customisation has been achieved. The 
problem of receiving “off-the-shelf programmes (instead of customised programmes) is 
mentioned several times. Business schools are often tempted to ‘sell’ standard elements in a 
customised programme. The interviewed sponsors give some suggestions for improvement. 
For example, the customised programmes should rely more on internal speakers and 
address more relevant topics and case studies. 

In terms of the changes in performance ratings as assessed by an internal performance 
system we have outlined in the previous chapter that no comprehensive analysis was 
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possible due to the changes in the corporate system at the time of the research (cf. 
subsection 4.2.3.2). However, we were able to collect some exemplary data in order to 
demonstrate the underlying idea of this measure. We would like to emphasise that this data 
is not representative and can thus only give a limited impression. 

One of the criteria, through which each executive’s performance is assessed, is strategic 
thinking. We have gathered the performance ratings for a handful of participants from each 
sample. In our analysis we have compared the ratings before and after the participation in 
the programmes. 



Sample 


Changes in the Performance Ratings of Strategic Thinking 

(Before vs. After the Programme) 




Total ‘n’ 


Increase 


Unchanged 


Decreased 


C-Sample 


9 


4 


2 


3 




100% 


44% 


22% 


33% 


NC-Sample 


5 


1 


1 


3 




100% 


20% 


20% 


60% 



Table 5.7: Changes in the Performance Ratings of Strategic Thinking 

Source: STEER - Administrative Data 



From table 5.7, we can see that the majority of executives who have taken part in the 
customised programmes improved in their rating of strategic thinking (four out of nine). In 
comparison three out of the five test participants from the NC-Sample have received lower 
ratings than before the programme. Hence, we captured an impression that the 
participants of the customised programmes were able to improve their performance ratings in 
strategic thinking, whereas the participants of the non-customised programmes experienced 
a decrease in their ratings. 

The extremely small test sample size though does not allow us to use the results of table 
5.7 as evidence. Nevertheless, we included the changes in the performance ratings in order 
to explain the general idea of the measure. From 2001 onwards, the new performance 
assessment system will be applied to all DaimlerChrysler executives. Then the analysis 
could be spread to a bigger sample. One suggestion for further research is therefore to redo 
the analysis of table 5.7 on a much larger scale. 94 



94 We acknowledge that using executive ratings for programme evaluation must be considered with care. The 
participation in a particular programme is certainly not the only variable that affects an executive’s rating at the 
end of the year. However, as mentioned before, the data would add another piece in the evaluation puzzle. 
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Summarising the stakeholder perspective - we discern a positive response for both types of 
programmes. The executives regard their participation in both programmes types as 
something extraordinary and rate their programme experiences above average. In terms of 
the value assessment, the customised programmes are rated a notch better than the non- 
customised programmes. The call back and cancellation rates portray the non-customised 
programmes in a better light. The sponsor attendance rate reflects one of the major 
advantages of customisation - direct internal input. Finally, the changes in executive ratings 
for strategic thinking have given us an interesting first impression. The test sample indicates 
that strategic thinking might be more positively influenced by customised programmes than 
by non-customised programmes. 

However, we cannot provide clear evidence that would confirm or reject our null hypothesis 
(cf. 4.2.2). Therefore, we continue our analysis with the results of the content perspective. 



5.1. 2.2 Results from the Content Perspective 

“Learning is like a language: A large share is passive knowledge, which sleeps 
until it is activated when you need it . " 



(Participant from the C-Sample) 

As devised by STEER, we distinguish between content statistics and feedback in the content 
perspective. 

5.1. 2.2.1 Content Statistics 

The content statistics comprise several assessments by former participants - relevance of 
content to DaimlerChrysler and to the job, learning with respect to strategic thinking, to the 
LEAD criteria and to the six corporate goals. Furthermore, we calculate the degree of 
customisation of the programmes’ agendas. 

First, we look at the relevance of content to DaimlerChrysler as assessed by participants. 
The results from figure 5.10 confirm the underlying idea of customisation (cf. Liedtka et al. 
1999). The participants of the C-Sample rate the degree of relevance of the content for 
DaimlerChrysler considerably higher (average 4.01) than the participants of the NC-Sample 
(average 3.52). The difference of these two ratings is significant at the five percent level 
(significance level: a ct oNc = 0.032). 
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C-SAMPLE 



Did the programme content respond to the challenges of 
DaimlerChrysler at the time? 



n = 72 




NC-SAMPLE 

Did the programme content respond to the challenges of 
DaimlerChrysler at the time? 




Figure 5.10: Rating of Relevance of Content to DaimlerChrysler 

Source: STEER - Follow-up Telephone Interviews 

Furthermore, we can see that the length of programmes has little effect on the relevance 
rating (trimmed NC-Sample average 3.57, significance level a C t 0 trimmed nc = 0.134). Internal 
programmes therefore offer a higher degree of perceived customisation of content to 
DaimlerChrysler than open programmes. 

The picture changes when we take a look at the rating of relevance of content to the 
participants’ jobs. In figure 5.11 we can see that the non-customised programmes achieve 
a substantially higher rating. 
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C-SAMPLE 



How relevant was the content of the programme to your 
job? 




NC-SAMPLE 



How relevant was the content of the programme to your 
job? 



100 % 
75% 
50% -I 
25% 
0% 



Very relevant 
| Trimmed (61%) 



0 4.15 

(Trimmed 0 4.22) 



53% 

26% 

' m ■ 





9% 


6% 


6% 


0% 


Mostly 


Medium 


Slightly 


Not very 


NO 


relevant 


(average) 


relevant 


relevant 


ANSWER 


(17%) 


(13%) 


(0%) 


(9%) 


(0%) 



Figure 5.1 1 : Rating of Relevance of Content to the Participants’ Jobs 

Source: STEER - Follow-up Telephone Interviews 



The difference between the C-Sample (average 3.78) and the NC-Sample (average 4.15) is 
significant (significance level: a ct oNc = 0.009). The same is true for the analysis with respect 
to the trimmed NC-Sample (significance level oc c to trimmed nc = 0.008). We regard these results 
as a confirmation, i.e. the customised programmes are designed for a corporate level 
regardless of specific functions. The non-customised programmes are often chosen to suit 
the specific work (functional) needs of the participants. Hence, the customised programmes 
do not address the specific functional needs of the participants as well as the non- 
customised programmes. 
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Additionally, we surveyed the participants’ ratings of the learning with respect to the 
different performance criteria, which are used in the organisation to assess the executives’ 
performances (the LEAD criteria). These ratings respond to the question in how far the 
programme has supported the learning with respect to the criteria. The criterion ‘thinking 
strategically’ has been of particular attention due to the orientation of our research on 
strategy process capability. The results are depicted in figure 5.12. 



C-SAMPLE 

Did the programme help you to think more strategically? 



n = 72 




NC-SAMPLE 




Figure 5.12: Participants’ Ratings of the Learning with Respect to Thinking Strategically 

Source: STEER - Follow-up Telephone Interviews 

In terms of strategic thinking we can see a substantial difference between the two 
programme types. The C-Sample achieves an average rating of 3.94 points, whereas the 
NC-Sample achieves 3.64 points (3.65 in the case of the trimmed NC-Sample). The 
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difference is significant at the ten percent level (significance level: a c to nc = 0.086) for the 
NC-Sample. We can therefore infer that the customised programmes enable a more effective 
learning with respect to strategy process capability in the eyes of participants. 95 

In figure 5.13 we can compare these results with the ratings of the other four performance 
indicators for executives within the organisation. Thinking Strategically is the only one with a 
significant difference in terms of the statistical analysis. 96 

The other four other criteria show smaller differences in the ratings between the two 
samples. Interestingly, Implementing Change and Sharing Knowledge receive better ratings 
in the NC-Sample than in the C-Sample. 



Comparison of LEAD Criteria 

n c = 72 
n N c = 34 



4 
3 
2 
1 
0 

Thinking Implementing Leading Sharing Creating Value 

Strategically Change People Knowledge 

Trimmed (3,65) (3,43) (3,00) (3,52) (3,57) 

■ C-Sample □ NC-Sample 



Figure 5.13: Rating of the Programme Content with Respect to the Five LEAD Criteria 

Source: STEER - Follow-up Telephone Interviews 
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Taking a look at the NC-Sample and trimmed NC-Sample we can see that the length of the 
programme has an impact on learning with respect to LEAD. The NC-Sample, which 
includes the longer programmes, achieves a higher rating in three of the five criteria. For the 
remaining two criteria, the two samples attain almost identical values. Length seems to play 
a role in terms of the degree of learning. 



95 We acknowledge that the term Strategic Thinking can be ambiguous to the interviewees. However, all 
executives are rated on this criterion in terms of their LEAD performance. Therefore, they have been informed 
beforehand at the implementation of LEAD, what Strategic Thinking includes. 

96 The Mann-Whitney U-Test results in the following as: Implementing Change a = 0.225, Leading People a = 
0.81 1 , Sharing Knowledge a = 0.472 and Creating Value a = 0.974 (all concerning C to NC). 
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Additionally, we have conducted a rating of the professional learning with respect to the six 
corporate goals of the corporate strategy of DaimlerChrysler. These goals were defined in 
the aftermath of the merger in 1998/9 and form an important part of the strategic mission and 
vision statement of DaimlerChrysler in 2000/2001. The strategic goals have been 
communicated broadly within the corporation. Therefore, the six goals represents a broad 
indication for the strategic direction of the company. 

In figure 5.14- we can see that the C-Sample generally achieves slightly better ratings on the 
learning with respect to the six goals - except in Integrating Operations. The most substantial 
differences can be discerned when we trim the NC-Sample. Comparing the C-Sample with 
the trimmed NC-Sample, we can see that the participants of the C-Sample rate the learning 
on four out of the six corporate goals substantially higher than the participant from the 
trimmed NC-Sample. However, only the difference in Integrating Operations is significant at 
the ten percent level (significance level a c to nc = 0.086; significance level a c to trimmed nc = 
0.052). 



Comparison of the Six Corporate Goals 

n c = 34 
riNc= 34 
5 

4 

3 

2 

1 

0 

Delighting Increasing Managing a Sustaining Integrating Going Global 

Customers Profitability Portfolio Growth Operations 

Trimmed (3,10) (3,38) (3.1 1) (3,40) (3,90) (3,84) 

■ C-Sample □ NC-Sample 



Figure 5.14: Rating of the Programme Content with Respect to the Six Corporate Goals 

Source: STEER - Follow-up Telephone Interviews 

These results are in line with the ratings on ‘thinking strategically’. Strategy knowledge in 
form of learning about the corporate strategy is mediated more extensively in the customised 
programmes. 

Furthermore, we assessed the degree of customisation of each programme type by using 
the following customisation code. We have sampled the programme agendas of the DCU 
programmes and applied the following key for assessing customisation. 



^_ 3 <52 3,50 ^ 3,60 3.45 _ — . H 

i in 1 1 
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Rating Code 


Degree of Customisation 


Internal Speaker/Presentation 


100% 


Topic on DaimlerChrysler 


75% 


Topic Related to the Industry 


50% 


(Strategic) Management Topics 


25% 


Any Topic 


12.5% 



Table 5.8: Rating Code for Assessing the Degree of Customisation 
Source: STEER 



In terms of the non-customised programmes we assume that the degree of customisation is 
located between Any Topic and Strategic Management Topics. Therefore, we have set the 
degree of customisation for the open programmes to 18.75 percent (the mean of 12.5 and 
25 percent). We can justify this degree of customisation in two ways. First of all, the content 
of the non-customised programmes is geared towards global companies. DaimlerChrysler is 
a global company and therefore some topics might be as if they were customised (i.e. they 
are specifically relevant). Secondly, DaimlerChrysler might serve as an example in the 
general business education, in particular due to the merger. This could be in form of a case 
study. Open programmes might talk about strategic DaimlerChrysler issues as well. 97 Table 
5.9 illustrates the rates of customisation calculated for the two programme types. 



Sample 


Estimated Degree of 
Customisation 


C-Sample 


46% 


NC-Sample 


18.75% 



Table 5.9: Estimated Degree of Customisation of Content 

Source: STEER - Administrative Data from DCU and Business Schools 



It is difficult to use the results for the comparison of closed and open programmes. The rate 
of the open, non-customised programmes is likely to be lower by our assumption. However, 
we found it interesting that the rate of customisation was not higher in the case of the 
internal programmes. It seems that the number of agenda items with high relevance to 
DaimlerChrysler (75% and 100%) do not represent the majority. Consequently, the overall 
rate is below 50%. We can infer that the degree of customisation could still be increased 
through more topics on DaimlerChrysler or the industry. Additionally, one could consider 
more internal speakers or presentations, but these must be integrated into the overall flow of 
the programme as well. 



97 The results of figure 5.10 (on the relevance to DaimlerChrysler) show that even the open programmes receive 
some positive ratings. 
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5.1. 2.2.2 Content Feedback 



The content feedback consists of the following elements. First of all, we take a look at the 
participants’ feedback concerning the skills, knowledge and mentality (attitude) learned. 
Subsequently, we deal with the strategic relevance as assessed by the sponsors. 

In the follow-up telephone survey we asked the former participants about their personal and 
professional learning with respect to skills. Table 5.10 represents the findings concerning 
the question of new or improved skills. 



C-Sam 


pie 


n=34 


Ranking 


“Which skills did you learn or improve?” 


Replies 


Percentage 


1 








2 




10 


ms 


3 


Finance skills 


10 




4 


Leadership skills 


10 




5 


Process and change management skills 


10 




6 


Interpersonal skills 


8 


24% 


7 


General management and business skills 


7 


21% 


8 


Intercultural skills 


6 


18% 


9 


Planning skills 


6 


18% 


10 


Analytical skills 


5 


15% 




Motivating and rewarding skills 


5 


15% 



NC-Sample 


n=34 


Ranking 


“Which skills did you learn or improve?” 


Replies 


Percentage 


1 


Communication skills 


10 


29% 


2 


Analytical and problem solving skills 


8 


24% 


3 


General management and business skills 


8 


24% 


4 


Process and change management skills 


8 




5 


Intercultural skills 


7 




6 




7 




7 




6 




8 




6 




9 




6 




10 


Marketing skills 


5 






No specific skills 


5 


15% 



Table 5.10: The Learning of Participants with Respect to Skills 

Source: STEER - Follow-up Telephone Interviews 



The results of table 5.10 support the higher rating of Thinking Strategically (for the C- 
Sample) in figure 5.12. Ca. 40 percent of the executives in the C-Sample stated that they 
have learned skills concerning strategy - compared to ca. 20 percent in the NC-Sample. The 
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difference in responses is another indication for the ability of the customised programmes to 
build strategy process capability. 



Overall, we can see that the two programme types are similar with respect to teaching 
general management skills. ‘Communication, finance, leadership, intercultural, interpersonal’ 
and ‘analytical skills’ are found in the top ten of both samples. 

Table 5.1 1 lists the responses on the knowledge gained during the programme. Once more, 
we find ‘strategy, finance, leadership’ and ‘general management knowledge’ on both lists. 



C-Sam 


! 

pie 


n=34 


Ranking 


“What knowledge did you gain?” 


Replies 


Percentage 


1 


General business orientation and management knowledge 


9 


26% 


2 


Internal knowledge and information on DaimlerChrysler 


9 


26% 


3 


Strategy knowledge 


8 


■i 


4 


Finance knowledge 


6 


msm 


5 




5 


■eh 


6 




4 


■eh 


7 




4 




8 




4 


■ESS 


9 




4 


■B^l 


10 


Strategic direction of the company | 


4 


12% 



NC-Sample 


n=34 


Ranking 


“What knowledge did you gain?” 


Replies 


Percentage 


1 


Finance knowledge 


9 


26% 


2 


General management knowledge 


9 


26% 


3 


Insights into other companies and their practices 


9 


26% 


4 


Intercultural knowledge 


9 


26% 


5 


Strategy knowledge 


7 


21% 


6 


Change management knowledge 


5 


15% 


7 


Competitive market knowledge 


5 


15% 


8 


Interpersonal knowledge 


5 


15% 


9 


Leadership knowledge 


5 


15% 


10 


Controlling knowledge 


4 


12% 



Table 5.1 1 : The Learning of Participants with Respect to Knowledge 

Source: STEER - Follow-up Telephone Interviews 



The C-Sample includes ‘internal knowledge and information on DaimlerChrysler’ as well as 
‘internal development and trends’ and the ‘strategic direction of the company’. Customised 
programmes thus mediate important corporate knowledge and play a crucial role in 
spreading strategy knowledge within the organisation. 
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In contrast to the C-Sample, the NC-Sample includes knowledge items that show the 
external character of non-customised programmes. The ‘insights into other companies and 
their practices’ as well as ‘intercultural knowledge’ are rated relatively high. From a strategy 
perspective, the external nature of the programme helps to build ‘competitive market 
knowledge’ and general ‘knowledge related to globalisation’. 



C-Sam 


pie 


n=34 


Ranking 


“Did the programme change your opinion or attitude?” 


Replies 


Percentage 


1 


Encouraged me and provided confidence that 1 am on the right track 


8 


24% 


2 


Became aware of aspects in controlling and finance 


6 


18% 


3 


Built awareness for strategy 


6 


18% 


4 


Broadened my horizons 


5 


15% 


5 


Only slight change in attitude 


4 


12% 


6 


Enabled me to put our business into a wider perspective through the programme 


3 


9% 


7 


Provided confirmation and consolidation of my knowledge and awareness 


2 


6% 


8 


Changed my way of thinking 


2 


6% 


9 


Enhanced my general business awareness and thinking 


2 


6% 


10 


Helped to develop and advance myself 


2 


6% 




Received new impulse for thinking and new perspectives 


2 


6% 




Strengthened my intercultural awareness 


2 


6% 



NC-Sample 


n=34 


Ranking 


“Did the programme change your opinion or attitude?” 


Replies 


Percentage 


1 


Only slight change in attitude 


9 


26% 


2 


Learned about personal integrity 


7 


21% 


3 


Strengthened my intercultural awareness 


7 


21% 


4 


Broadened my horizons 


6 


18% 


5 


Improved global and holistic business awareness 


6 


18% 


6 


Reflected on the situation of our company 


6 


18% 


7 


Reinforced my self-confidence 


6 


18% 


8 


Took new perspectives and insights 


6 


18% 


9 


Became aware of aspects of the Human Resource Management 


4 


12% 


10 


Built motivation to improve continuously 


4 


12% 




Compared our company with other companies 


4 


12% 




Confirmed my skills and knowledge 


4 


12% 




Improved my strategy skills and thinking 


4 


12% 




Learned and practised openness 


4 


12% 



Table 5.12: The Learning of Participants with Respect to Changing an Opinion or Attitude 

Source: STEER - Follow-up Telephone Interviews 



The third component of strategy process capability is mentality (cf. 2. 2. 1.1). We consider the 
question on attitude changes as a difficult one in terms of responding to it. Participants (like 
everybody of us) are often not aware of the sub-conscious changes in mentality or attitudes. 
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Yet, in line with our understanding of strategy process capability we decided to pose the 
questions in the interviews. 

The red line of the responses in table 5.12 (on the previous page) is the ‘building of 
awareness ’, which can be found in both groups. The executives leave the seminar with a 
new focus towards certain business aspects. Furthermore, the aspect of ‘refreshment’ and 
‘reconfirmation’ of one’s own skills and knowledge plays a visible part. For some executives 
the programme acts as a ‘boost in self-confidence’. 

As we can see from the results, one advantage of the NC programmes is the opportunity to 
'reflect on one’s own company’ due to learning together with executives from other 
companies. Almost 20 percent of the NC-Sample responded in that way, compared to only 
three percent in the C-Sample. 

Overall, we observe many similarities in the responses on skills, knowledge and mentality. 
Customised and non-customised programmes seem to provide the same types of general 
management skills. Moreover, in terms of mentality both types create foremost awareness, 
confirmation and motivation. The most significant difference can be read off the responses 
on knowledge. Here, we find that the customisation impacts the programme content visibly 
and thus creates a stronger focus on the organisation itself. 

We come to the strategic relevance of content as assessed by the sponsors of the 
executive programmes. Unfortunately, we were unable to conduct as many interviews as 
originally planned. Nevertheless, we extracted some general responses concerning the 
importance of management development programmes. 

‘‘The DCU has established a portfolio of customised programmes with world-class 
business schools, which prepare our top executives with the required skills for 
realising strategic objectives quickly and in accordance with top quality 
standards. ” 



(Sponsor interview) 

The quote above reflects a general satisfaction with the work of DCU’s Executive Education, 
i.e. a very positive feedback for the customised programmes. The sponsor recognises that 
the DCU has succeeded in setting up customised programmes, that provide a benefit to the 
executives of the company. The next quote addresses the aspect of customisation more 
specifically. 
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“We don’t place an exclusive emphasis on the expertise offered by university 
professors and consultants - the integration of the know-how and skills found in 
our company is also a vital component in the development of customised 
programmes in order to fit our specific requirements. 

Our programmes employ a variety of speakers from all levels of the organisation. 

Thus we can pursue our objective of creating a shared understanding on the long 
and short term issues of DaimlerChrysler among our executives. ” 

(Sponsor interview) 

We can see in the second quote that customisation is regarded as the key to success of the 
internal DCU programmes. Thereby, customisation is created by employing corporate 
expertise through internal speakers. The sharing of organisational knowledge and best 
practices is regarded as an important value driver. The importance of customised, i.e. 
relevant, content of the internal programmes can be seen in the next quote. Here, a sponsor 
emphasises the need for focusing on content, which is relevant for DaimlerChrysler. 

“[The executive programmes] must have - next to the more general references - 
a very concrete relevance for our business. This implies, that we should not deal 
only with DaimlerChrysler by force, but for example that we deal more with 
business cases from the automotive supplier industry or with business cases 
about our competitors. ” 



(Sponsor interview) 

The quote above supports our assessment of the degree of customisation (tables 5.8 and 
5.9). Customisation does not only imply internal speakers or DaimlerChrysler as a topic “by 
force”. However, it should include cases from the automotive industry or cases on 
DaimlerChrysler’s competitors. 

The fact that this sponsors demands an even higher degree of customisation confirms our 
result of the 46 percent customised content in table 5.9. The closed programmes of the DCU 
are customised to some extent, but there is still room for improvement. 

Overall, the feedback of the corporate sponsors has been positive. All of them supported the 
idea of having customised programmes - in particular with respect to transferring first-hand 
corporate messages and information. All sponsors defined the role of customised 
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programmes as an important one with respect to the integration of the organisation, i.e. 
through informal networking. 

Summarising the results of the content perspective - our application of STEER has 
confirmed the common expectations about customisation (Liedtka et al. 1999). Customised 
programmes show a higher perceived relevance of content with respect to important 
corporate topics and the organisation on the whole. 

The content of the non-customised programmes, however, seems to be more job-related. 
Thereby, the content perspective supplies the first statistical evidence that the two types of 
programmes are different and that the null hypothesis might be wrong, i.e. should be 
rejected. 

In terms of strategy process capability, the customised programmes have achieved a 
higher score with respect to the quantitative as well as the qualitative measures. The 
quantitative results have been confirmed as significant by our statistical analysis. We can 
thus infer that customisation enables a stronger development of strategy process capability. 



5.1. 2. 3 Results from the Process Perspective 

“The right mix of participants is important because the exchange of experiences 
is the vital part of such management development programmes. ” 

(Participant of the C-Sample) 

The third perspective of STEER is the process perspective. As suggested in subsection 

3. 3. 2.3 we distinguish centrally gathered data from de-centrally gathered information. 

5.1. 2.3.1 Centrally Gathered Data 

We have applied the process indicators of STEER to both samples. In terms of the centrally 
gathered data, we analysed totals and averages of programmes realised , target group share 
in the programmes , productivity ratios of participants trained per member of staff and 
programme capacities used. 

First of all, we compare the totals and averages of programmes realised in table 5.13 (on 
the next page). As expected the non-customised programmes offer a much larger pool of 
programme types than the customised ones. 
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Sample 


Types of Programmes 


Total Number of 
Programmes 


Rate of Programmes 
Per Year 


C-Sample 


4 


14 


4.7 


NC-Sample 


22 


29 


4.8 



Table 5.13: Totals and Averages of Programmes Realised 

Source: STEER - Administrative Data from DCU and Business Schools 



The rate of programmes per year cannot be used for an objective comparison because the 
rates of the two samples depend on different factors. The rates show that the DCU runs ca. 
five programmes in co-operation with Harvard, INSEAD and IMD per year. At the same time, 
executives from DaimlerChrysler in general have visited ca. five programmes per year at the 
respective business schools as well. 

Secondly, we have calculated the target group share of the programmes. The underlying 
rationale for the target group share is twofold. First of all, the programme is designed for a 
specific level in terms of its content and methodology. For example, a programme for CEOs 
is likely to be different than a programme for middle managers. We assume that it is 
therefore of value to address the right group of participants with the right topics. Secondly, 
it is more valuable for participants to meet their equals in order to share experiences and to 
network. Consequently, we conclude that a higher share of the target group level is more 
valuable than a lower share. 



Table 5.14 summarises the results of our calculations concerning the target group share for 
the two populations. 



Population 


Programmes 


Target Group 


Number of Target 
Group Members 


Target Group Share 
in the Programmes 


C-Population 


Harvard (166 execs.) 


Vice Presidents 


67 


40% 


(N = 418) 


INSEAD (169 execs.) 


Directors 


87 


51% 




IMD (83 execs.) 


Senior Managers 


72 


87% 




Weighted Average 






54% 


NC-Population 


(see programme 
descriptions in 
appendix 8.11) 


Vice Presidents 


' 7 


43% 


(N = 41) 


Directors 


14 


82% 


(NA = 4) 


Senior Managers 


4 


100% 


(Relevant N = 37) 


Weighted Average 






67% 



Table 5.14: Target Group Share in the Programmes 

Source: STEER - Administrative Data from DCU and Business Schools 
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We have compared the programmes’ descriptions with the actual levels of participants and 
established how many of the participants matched the intended target group level. In 
general, the non-customised programmes achieve a higher target group share. 67% of 
the participants from NC-Sample attended a programme that corresponded with their level 
and responsibility (compared to only 54 percent in the C-Population). This result is interesting 
as one would expect that the internal programme administrators would have more 
information about the potential participants than the external programme organisers. It is also 
in line with the cancellation rates (table 5.4). It seems that the customised programmes have 
not yet used their insider advantage to achieve a better match of intended and actual target 
group. 

The target group share illustrates another phenomenon. We can see that is generally more 
difficult to motivate the higher executive levels to attend a programme. In both populations 
the target group share for Vice Presidents was considerably lower than for the other levels. 

Thirdly, we suggested to include a productivity ratio in the process perspective of STEER. 
One possibility is to analyse the ratio of participants trained per member of staff. The ratio 
illustrates the number of participants per year divided by the number of staff who are involved 
in the designing, organising and running the programmes. The non-customised programmes 
do not rely on members of DCU staff - the application is done by the respective participants 
themselves. 



The results of the comparison on the productivity ratio is portrayed in table 5.15. 



Population 


Total Number of 
Participants 1998 to 2000 


Number of Staff Involved 
in Programme 
Organisation 


Number of Participants 
Trained Per Member of 
Staff 


C-Population 


418 


4 


104.5 


NC-Population 


17 


None 98 

1 


Unlimited 99 



Table 5.15: Participants Trained Per Member of Staff 

Source: STEER - Administrative Data from DCU and Business Schools 



Table 5.15 shows the advantage of non-customised programmes. The organisation of the 
external programmes is outsourced completely and thereby a high degree of flexibility is 
achieved. In the case of the customised programmes the frame of activities is more limited in 

98 The non-customised programmes are organised by the participants themselves. Therefore, corporate HR staff 
is involved only to register the human resource development activity. There is no full time HR staff to handle the 
participation in non-customised programmes. 
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comparison. It is more difficult to increase (or decrease) the number of participants on a short 
term basis because of the given capacities of the DCU staff. However, this productivity ratio 
should be put into perspective to content and cost measures (see content and business 
perspectives). Furthermore, we should not forget that the DCU works on the basis of a 
virtual structure. The realisation of programmes is already outsourced and the DCU 
consists of a mere core team. A fully staffed training department, including campus and 
faculty, would probably be even less flexible in terms of changes in demand. 

Finally, we have considered the programme capacities used. Among the different process 
indicators, the capacity usage rate represents one of the most objective measures for the 
process efficiency of running the programmes. Furthermore, it can serve as an indicator for 
the general popularity, i.e. success, of executive education programmes. Calculated over 
several years, the results in table 5.16 account for fluctuations in demand, which can occur 
on the internal as well as on the external market. 100 



Population 


Scheduled Capacity 


Actual Capacities Used 


Capacity Usage Rate 


C-Population 


525 


418 


79% 


NC-Population 


Data not available 


Data not available 


70% 101 



Table 5.16: Programme Capacities Used 

Source: STEER - Administrative Data from DCU and Business Schools 



Table 5.16 shows that the DCU programmes achieve a better capacity usage rate. 
However, the data on the usage rate should be considered with care. The rate of the 
external, non-customised programmes stands for the general rate of the respective business 
schools, i.e. many more programmes and events than the DCU programmes. Our objective 
for showing the data is again to explain the underlying idea. The respective process 
situations of customised and non-customised programmes are different, and it would be 
better to compare the DCU programmes to other programmes within DaimlerChrysler. The 
only conclusion from table 5.16, that we can draw, is that the DCU does not use its full 
potential currently. Thereby, ca. 20 percent of its potential value creation is lost. We suggest 
to tackle the problem of underused capacity through stricter cancellation procedures (see 
above), or through a more adequate registration process. For example, if we know from 



99 We assume that the open market is able to adjust more quickly to changes in demand due to global capacity. 
Even if the three business schools are ‘full’, there are other schools which could host additional participants. 

100 We presume that an unpopular programme is altered/stopped over the years, so that over the total period of 
analysis we can assume sufficient demand to fill all the programmes. 

101 The capacity usage rate for the external, non-customised programmes was not measured by us, but resulted 
from interviews with the controlling staff at the respective business schools. 
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experience that 15 percent of the scheduled participants cancel, one could over-register the 
programme like airlines do. Additionally, one could use a waiting list system. 



5.1. 2.3.2 De-Centrally Gathered Data 

In the framework of the process perspective we gathered a selection of qualitative data de- 
centrally through telephone interviews. We investigated the strengths and weaknesses as 
perceived participants. Furthermore, we include responses on the aspect of customisation. 

The strengths as perceived by participants enable a retrospective assessment of the 
positive aspects of the programme. The collected responses in the telephone interviews are 
listed in table 5.17. 



C-Sam 


pie 


n=72 


Ranking 


“What were the strengths of the programme?” 


Replies 


Percentage 


1 


The knowledge and competence of the professors 


42 


58% 


2 


The composition and diversity of participants with respect to their personal and 
functional backgrounds 


28 


39% 


3 


The selection of relevant topics with respect to their variety and composition 


27 




4 


The opportunity to network with colleagues within DaimlerChrysler 






5 


The case study method and the practical orientation of the content 


19 


Km 


6 


The exchange of experiences among participants 


18 


ksk 


7 


The learning facilities and environment 


16 




8 


The cross-functional learning 


14 


19% 


9 


The professionalism of the programme with respect to its organisation 


14 


19% 


10 


The interaction with other participants 


13 


18% 




The interesting and stimulating content 


13 


18% 



NC-Sample 


n=34 


Ranking 


“What were the strengths of the programme?” 


Replies 


Percentage 


1 


The knowledge and competence of the professors 


19 


56% 


2 


The composition and diversity of participants 


18 


53% 


3 


The exchange of experiences among participants 


14 


41% 


4 


The professional relevance of the content 


10 


29% 


5 


The internationality of the programme 


9 


MW 


6 


The learning facilities and environment 


9 


Km 


7 


The academic quality of the teaching 


8 


— 


8 


The intensive curriculum 


8 


K3H 


9 


The case study method 


7 


Km 


10 


The external speakers 


7 






The intercultural learning 


7 


21% 




The selection of topics 


7 


21% 



Table 5.17: The Strengths of the Programme as Seen by the Participants 

Source: STEER - Follow-up Telephone Interviews 
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Overall, we can observe that the ‘ knowledge and competence of professors’ is perceived as 
the most important aspect by both participant groups. The ‘composition and diversity of 
participants’ as well as the ‘relevance of content’ follow with similar rankings. Both groups 
emphasise the importance of ‘exchange among participants'. 

The most striking differences can be found in the importance of ‘networking’ and the ‘cross- 
functional learning’. The ‘opportunity to network’ ranks fourth place in the C-Sample (29 
percent), but only 13th place in the NC-Sample (18 percent). ‘Cross-functional learning’ is 
mentioned by 19 percent of participants from the C-Sample, but only by nine percent of the 
NC-Sample. On the other hand, the ‘learning from other companies’ is mentioned by 15 
percent of participants interviewed in the NC-Sample. The 'internationality of the programme’ 
is considered more often by the members of the NC-Sample. The aspect ranks fourth place 
in the NC responses (26 percent), but only 19th place among the C-responses (13 percent). 



C-Sam 


pie 


n=72 


Ranking 


“What were the weaknesses of the programme?” 


Replies 


Percentage 


a 


Missing emphasis on discussion and interaction about relevant corporate issues 
and problems 


27 


38% 




Poor selection of case studies - irrelevant to DaimlerChrysler 


18 


25% 


3 




18 


25% 


4 


Overwhelming reading materials - too much to prepare and digest 


13 


18% 


5 


Lacking consistency of the programme with respect to its quality 


12 


17% 


6 


Little relevance of content to our business 


mm 


17% 


7 


Missing practical relevance of topics with respect to my responsibilities 




17% 


8 


No perceived weaknesses 


12 


17% 


9 


Missing grounding of the programme with respect to our corporate realities 


11 


15% 


10 


Little practical learning that can be transferred to real life 


7 


10% 




Too theoretical/academic focus of the content 


7 


10% 



NC-Sample 


n=34 


Ranking 


“What were the weaknesses of the programme?” 


Replies 


Percentage 


1 


Missing relevance of content to my responsibilities 


11 


32% 


2 


Overwhelming materials and intensity of the programme 


10 


29% 


3 


No perceived weaknesses 


7 


21% 


4 


Insufficient focus on our business of the open programme - too general 


6 


18% 


5 


Wrong selection of participants with respect to their functional and personal 
backgrounds 


6 


18% 


6 


Uneven quality of professors - some were not very good/disappointing 


5 


15% 


7 


Inappropriate timing of the programme (too long) 


3 


9% 


8 


No free time to network informally 


3 


9% 




(skipped responses 9 to 10 because they have only two entries) 







Table 5.18: The Weaknesses of the Programme as Seen by the Participants 

Source: STEER - Follow-up Telephone Interviews 
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In terms of the perceived weaknesses by participants we have gathered the responses 
presented in table 5.18 (on the previous page). 

Starting with the similarities of the two samples, we can find several weaknesses mentioned 
by both samples. Some participants complain about the ‘overwhelming reading material’ and 
the ‘uneven quality of professors’. These aspects seem to be common quality problems of 
both types of programmes. However, the differences in perceived weaknesses are more 
interesting. 

In the C-Sample four of the top ten weaknesses concern the ‘missing relevance or focus on 
DaimlerChrysler’ and on corporate issues. These responses indicate that for many 
participants the degree of customisation has not yet been high enough. They demand 
more relevance in terms of method (‘missing emphasis on discussion and interaction’), 
content (‘little relevance of content), practical examples (‘poor selection of case studies), 
and tone (‘missing grounding of the programme with respect to our corporate realities’). 

Here, we can see how the different perspectives of STEER serve as a cross-validation. 
These results of missing relevance validate our findings in the content perspective, i.e. the 
calculated degree of customisation and the responses by sponsors. 

The responses from the NC-Sample also state that there is room for improvement. About 
one-third said that there was a ‘missing relevance of content to their responsibilities’. 
Furthermore, 18 percent criticise the ‘insufficient focus on the DaimlerChrysler business’. We 
consider these 18 percent as an interesting indicator for the value of customisation. Even in 
the setting of an external management development programme, participants consider it as 
important for their learning to refer to their business/company. 

In our analysis we have come across statements, which stress that general learning can be 
transferred to one’s business. For example, one participant said that he took the learning 
from a “Boeing Case” and applied it successful to his automotive business. However, an 
important share of participants want to have the transfer made for them, i.e. they would like 
to see a clear reference to their personal/professional situation. As we can see, the 18 
percent of the NC-Sample consists of these participants. 

Overall, a large share of the criticism of both samples concerns the content of the 
programmes. In particular, in the case of the customised programmes, more relevance and a 
clearer link to the business is demanded. 
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C-Sample 


n=34 


Ranking 


“What would have been different, if the programme had been ‘open’ 
with respect to its participants?” 


Replies 


Percentage 


1 


Would not have been as open with respect to strategy and internal processes - a 
main advantages of the closed programme 


16 


47% 


2 


Consider it best to stay in a closed group 


10 


29% 


3 


Would enable new and different input from other organisations dealing with the 
same problems and challenges 


8 


24% 


4 


Would not have provided the same networking opportunities 


8 


24% 


5 


Would enable a broader mix of participants with respect to business areas and 
functions 


6 


18% 


6 


Would have been freed from the complete fixation on DaimlerChrysler 


6 


18% 


7 


Would have transmitted a broader knowledge from cross-company and cross- 
industry learning 


6 


18% 


8 


Would not have been able to learn from each other internally 


5 


15% 


9 


Would not have ensured the same speed with respect to developing team spirit 
and cohesion 


5 


15% 


10 


DaimlerChrysler already offers a great variety and diversity of businesses and 
functions 


4 


12% 




Would have been less valuable because it would be missing the in-house 
corporate communication and speakers 


4 


12% 




Would have been more difficult to communicate because we would not speak the 
same language 


4 


12% 




Would have shown us the external perspective of how other people and firms 
perceive us 


4 


12% 




Would not have managed to open up and discuss things freely and critically 


4 


12% 



NC-Sample 


n=34 


Ranking 


“What would have been different, if the programme had been 
‘closed’ with respect to its participants?” 


Replies 


Percentage 


1 


Would not have received new input and perspectives from other firms 


17 


50% 


2 


Would have lacked the breadth of personal and professional perspectives and 
experiences 


12 


35% 


3 


Would have ensured a better focus on relevant topics for DaimlerChrysler 


11 


32% 


4 


Would have been too concerned with DaimlerChrysler issues compared to an 
open programme 


10 


29% 


5 


Would not have achieved the cultural diversity than an open programme 


8 


24% 


6 


Could have better targeted and solved the problems of our business 


6 


18% 


7 


Would have been intellectual in-breeding with no new ideas from the outside 


6 


18% 


8 


Would not have benefited from the external expertise of other firms 


6 


18% 


9 


Would not have learned as much from the other participants as in an open 
programme 


5 


15% 


10 


Could be customised to the functional responsibilities of its participants 


4 


12% 




Would be less international and not cover all the cross-sectional aspects - not 
gaining the useful external knowledge 


4 


12% 




Would have been more politically restrained through the corporate culture 


4 


12% 




Would have created a more effective networking opportunity 


4 


12% 




Would not have achieved the same openness than the non-customised 
programme 


4 


12% 




Would offer a better opportunity to select the executives 


4 


12% 



Table 5. 1 9: Assessment of Customisation/Non-Customisation 

Source: STEER - Follow-up Telephone Interviews 
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We come to a more general process question of whether programmes should be customised 
or not. This customisation aspect has not been a part of STEER, but considering the 
research focus we have added a selection of questions to the telephone interviews. 

In terms of customisation we asked participants, what would have been different if their 
programmes would have been open (for customised programmes) or closed (for non- 
customised programmes). This ‘what if ...’ question (table 5.19 on the previous page) had the 
purpose to surface some of the underlying perceptions and values with respect to 
customisation. We thereby confronted participants of both samples with the opposite 
alternative in order to capture their responses. 

The responses from both samples reflect many of the assumed advantages and 
disadvantages (cf. section 2.2.2). The customised programmes offer more 4 confidentiality ’, 
better ‘intra-company and cross-functional learning’ and more effective ‘in-house 
communication’. The disadvantages of customised programmes are the advantages of the 
non-customised ones. Participants of the NC-Sample value the ‘new perspectives and 
external ideas, more professional and cultural variety’ and a more ‘open and global 
approach’. 

However, some responses are unexpected and thus reveal the underlying dynamics of 
customisation. The ‘networking’ aspect plays an important role. The ‘speed of developing a 
team-spirit’ is higher in internal programmes, i.e. in getting to know each other is more 
effective in the customised programmes. Furthermore, customised programmes provide a 
better opportunity to address problems and challenges openly in order to come up with 
solutions during the programme. 

On the other hand, we observe that for some participants customisation implies ‘intellectual 
confinement’ to corporate issues. These participants value the openness and the new 
inspiration of external programmes. Participants of the C-Sample speak of the disadvantage 
of the ‘complete fixation on DaimlerChrysler’, which some members of the NC-Sample call 
‘intellectual inbreeding’. Predefined corporate roles and culture are considered as a 
hindrance to openness and learning. 

We analysed the qualitative data of this questions in order to portray the general opinions of 
the two samples. In figure 5.15 (on the next page), we can see that most participants 
consider their programme form as the most appropriate. These results suggests that the 
overall population of executives is divided by the perceived advantages of customisation. 
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The two samples could thus present a case of adverse selection (Milgrom & Roberts 1992). 
Those executives, who prefer to learn in a non-DaimlerChrysler environment, would apply 
predominately to non-customised programmes, and vice versa. 



C-SAMPLE NC-SAMPLE 



Overall Opinion on Customisation - Seen by 
Participants of Customised Programmes 




Both types have For For open 

advantages and customisation/ programmes/ 

disadvantages Against open Against 

programmes customisation 



Overall Opinion on Customisation • Seen by 
Participants of NON-Customised Programmes 




Both types have For For open 

advantages and customisation/ programmes/ 

disadvantages Against open Against 

programmes customisation 



Figure 5.15: General Opinions on Customisation 

Source: STEER - Follow-up Telephone Interviews 



However, we do not deny, that psychological reasons might have influenced these 
responses as well. Interviewees might have had the feeling that they needed to justify their 
participation in the respective programme type. Most participants would therefore rate their 
programme experiences as the better one. 

Summarising the process perspective - the differences of the two programme types have 
become apparent. Thereby, the results of the process perspective confirm the evidence 
against the null hypothesis, i.e. several pieces in the evaluation puzzle indicate that 
customisation causes a perceivable and valuable difference. 

In the case of the non-customised programmes, all administration processes are outsourced 
to the respective business schools. From a process view, this implies more flexibility and 
variety. We find out more about the repercussions of this flexibility on costs in the business 
perspective. 

In terms of the specific strengths and weaknesses - important differences have surfaced. In 
particular, the process perspective confirms the importance of company-specific content and 
methods. However, the analysis through STEER has revealed that the customised 
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programmes have not yet brought to bear many of their advantages. The internal processes 
should allow a better organisation of programmes in terms of capacities used and reduction 
of cancellations. Foremost the internal programmes should be able to ensure more 
homogenous participant groups in terms of management level and responsibility. 

Overall, the process perspective adds new insights to the aspect of customisation. The 
results of the previous two perspectives are confirmed in several aspects. In particular, it 
seems that the degree of customisation could be higher. Furthermore, the process 
perspective has demonstrated potential disadvantages of customisation. 



5.1. 2.4 Results from the Business Perspective 

“The programme has helped me in the international strategy-making process. I have 
been able to apply my newly gained long-term thinking with respect to our regions” 

(Participant of the C-Sample) 

In the last of the four perspectives we distinguish between Accountable Costsl Financial 
Benefits and Organisational Outcomes. 

5.1. 2.4.1 Accountable Costs and Financial Benefits 

In the following subsection, we include a value rating of the programmes for DaimlerChrysler 
by the participants. Furthermore, we analyse the cost structures of the programmes and the 
willingness to pay for the programmes. And finally, we take a look at what participants have 
done differently after the programme. 

In line with the question for the personal value of the programme (cf. figure 5.8), we have 
asked former participants to perform a value rating of the programmes for 
DaimlerChrysler. The results are depicted in figure 5.16 (on the next page). 

We can see that the participants of the C-Sample rate the value of the programme higher 
than the participants of the NC-Sample - and substantially higher than the participants of 
the trimmed NC-Sample. The difference of the ratings is significant at the ten percent level 
(significance levels a ct oNc = 0.103). The difference between the C-Sample and the trimmed 
NC-Sample is significant even at the five percent level (significance level a c to trimmed nc = 
0.041). 
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C-SAMPLE 

Please rate the value of the programme for 
DaimlerChrysler in general. 




Very High High Medium Low No Value NO 

ANSWER 



NC-SAMPLE 



Please rate the value of the programme for 
DaimlerChrysler in general. 




Very High High Medium Low No Value NO 

ANSWER 

Trimmed (17%) (39%) (30%) (13%) (0%) (0%) 



Figure 5.16: Rating of the Programme Value for DaimlerChrysler 

Source: STEER - Follow-up Telephone Interviews 



In order to provide a better overview of all value ratings, we have compared the rating of the 
personal value with the rating of the value for DaimlerChrysler. Table 5.20 illustrates the 
results of our analysis. 



Sample 


Average Value Rating for 
‘You Personally 1 


Average Value Rating for 
'DaimlerChrysler’ 


C-Sample 


4.43 


4.07 


NC-Sample 


4.32 


3.74 


Trimmed NC-Sample 


4.22 


3.61 



Table 5.20: Comparison of Value Ratings 

Source: STEER - Follow-up Telephone Interviews 
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The value ratings of both samples are similar with respect to the personal value. Here, the 
customised programmes have a slightly higher rating. In terms of the value for 
DaimlerChrysler the participants’ judgement is clearer. The customised programmes are 
considered more valuable for the organisation than the non-customised ones. The average 
rating of the trimmed NC-Sample in that respect is 0.46 points lower than the one of the C- 
Sample. 

It is interesting to see that participants of both samples consider the personal value higher 
than the value for the company (the difference is 0.36 for the C-Sample and 0.61 for the 
trimmed NC-Sample). These results carry the more general message that executive training 
and development is considered foremost as a personal benefit. Only five participants (out of 
72) of the C-Sample rated the value for the company higher than the personal value. None of 
the non-customised programmes did. 

When dealing with the financial benefit of programmes the cost structure plays an important 
role. In table 5.21 we compare the actual costs of the programmes per participant per 
week. The exact cost information was considered confidential by the company. We have 
therefore used a relative comparison (we have taken the actual costs of the C-Sample as the 
reference point of a 100 percent). Furthermore, we have excluded any travelling expenses. 



Sample 


Relative Cost Comparison 


C-Sample (Reference point = 100%) 


100% 


NC-Sample 


101% 


Trimmed NC-Sample 


110% 



T able 5.21 : Relative Cost Comparison 

Source: STEER - Administrative Data from DCU and Business Schools 



We observe that the non-customised programmes are only marginally more expensive. 
However, we considered it necessary to compare the C-Sample with the trimmed NC- 
Sample due to the distortions through the lengths of the programmes. The longer 
programmes in the NC-Sample would lower the average as they tend to be cheaper on a per 
week comparison (on the basis of economies of scale). Comparing the costs of the C- 
Sample with the trimmed NC-Sample, we find that the non-customised programmes tend to 
be ca. ten percent more expensive than the customised ones. 

With respect to the cost discussion we would like to point out again that the programme costs 
might account only for a fraction of the total costs (Van Adelsberg 1999). 
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As an overall result of the cost comparison, we can state that the differences are not so 
significant. We cannot infer that one programme type is clearly cheaper than the other type. 
Consequently, we are unable to comment on a possible financial motivation to run 
programmes internally or externally. General economies of scale could be the reason for 
internal programmes, whereas the absence of fixed costs/overhead could favour external 
programmes. 

Additionally, we have conducted an experimental question. We have asked former 
participants of the C-Sample how much they would be willed to pay for the programme out of 
their budgets. 102 We assume that most business units/departments of the former participants 
have the financial means to pay for training and development. 

The assumption is not unfounded - many local training interventions are borne by the 
business unit or department. 103 In table 5.22 we illustrate the results with respect to 
participants willingness to pay. We omitted the real dollar values and substituted them 
with relative numbers again. 



Sample 


Participants Willingness to Pay 


C-Sample (Reference point = 100%) 


100% 


Actual Costs C-Sample 


90% 


Actual Costs NC-Sample 


91% 


Actual Costs Trimmed NC-Sample 


99% 



Table 5.22: Willingness to Pay 

Source: STEER - Follow-up Telephone Interviews 



The findings of the willingness to pay by participants are in line with the results of the value 
ratings of table 5.21. Participants of the customised programmes are generally willed to pay 
a higher amount than the actual costs of the programmes. Thereby, the amount mentioned 
by the members of the C-Sample is almost identical to the actual costs of programmes of the 
trimmed NC-Sample, i.e. the participants of the customised programmes stated almost 
exactly the prices of external, non-customised programmes. 



102 In the case of the NC-Sample we know what they were willed to pay since they did paid the programmes from 
their own budgets. 

103 We would like to remind the reader that the participants of the customised programmes do not have to bear 
the programme and accommodation costs, i.e. they are responsible only for their travelling expenses. 
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In order to understand the responses, we have subsequently asked participants how they 
have arrived at their responses. We have found that many executives possess a lot of 
experience with the prices of training and development in general. The willingness to pay can 
therefore be regarded as a well-founded assessment by participants. 104 The fact that most 
executives stated a slightly higher amount than the actual variable costs shows that many 
participants value the programme highly also in financial terms. In very abstract terms, we 
could say that the customised programmes have created a perceived financial surplus of 
ten percent over their actual costs. However, we suggest some more rigorous research 
before this figure can be used as solid evidence. 



The last aspect of the financial benefits is the transfer to business, i.e. what participants 
have done differently as a result of the programme. We posed this question to participants 
after several months - a period of time when they had sufficient time to implement the 
learning from the programme. The responses to the questions are found in table 5.23. 



C-Sample 


n=72 


Ranking 


“What have you done differently as a result of the programme?” 


Replies 


Percentage 


1 


Thought more strategically, rather than tactically 


13 


18% 


2 


Used the learning to improve and advance a project in my area of responsibility 


9 


13% 


3 


Created tangible results for my business unit 


8 


11% 


4 


Communicated/convinced people more effectively 


7 


10% 


5 


Was more knowledgeable and confident with respect to financial analysis 


7 


10% 


6 


Had a better understanding of the general business management 


5 


7% 


7 


Nothing in particular 


5 


7% 


8 


Was more sensitive when dealing with intercultural aspects 


5 


7% 


9 


Applied the concept of ‘value added’ to my business area 


4 


6% 


10 


Improved our supply chain management 


4 


6% 


11 


Introduced new financial performance indicators 


4 


6% 


12 


Built a better understanding of the strategic impact and consequences of my 
decisions 


3 


4% 


13 


Changed the organisation structures 


3 


4% 


14 


Felt better prepared for a new job 


3 


4% 


15 


Felt refreshed and confirmed in my skills and knowledge for my daily 
responsibilities 


3 


4% 




Made and implemented a new strategy 


3 


4% 




Planned differently - more long-term oriented 


3 


4% 




Thought differently about my job and my organisational role 


3 


4% 




Understood better our competitive situation 


3 


4% 



(continued on the next page) 



104 Furthermore, the executives interviewed make financial decisions daily and therefore their monetary 
judgement is trained. 



227 




NC-Sample 


n=34 


Ranking 


“What have you done differently as a result of the programme?” 


Replies 


Percentage 


1 


Made better decisions due to my professional learning 


9 


26% 


2 


Changed and improved internal processes and structures 


7 


21% 


3 


Communicated better 


6 


18% 


4 




6 




5 




6 


■ESS 


6 




5 


■ESS 


7 




4 


■ESS 


8 


Improved my change management 


4 


■ESS 


9 


Made and implemented a new strategy 


4 


■ESS 


10 


Thought more strategically, rather than tactically 


4 


12% 


11 


Was able to manage business contacts more effectively/efficiently 


4 


12% 


12 


Was more sensitive when dealing with intercultural aspects 


4 


12% 


13 


Approached problems and issues differently 


3 


9% 


14 


Benchmarked the learning and experiences of other companies 


3 


9% 


15 


Nothing in particular 


3 


9% 



Table 5.23: Transfer to Business 

Source: STEER - Follow-up Telephone Interviews 



In general, the results are quite varied, which explains the relatively low percentages of 
common responses. However, we can discern a tendency in the C-Sample towards an 
application of strategy process capability. Several responses show the more strategic 
approach of executives as a result of the programme. In the responses of the NC-Sample 
strategy appears not as frequently. 

Furthermore, we observe the application of other general management learning, like 
‘financial analysis’, ‘change management', ‘intercultural leadership’ and ‘communication’. We 
can see that the external programmes left their mark as well. Ca. 20 percent of the 
participants gained a new ‘external perspective’ on their company. A lot of executives refer to 
the ‘broadening of their horizons’, a ‘changed awareness' and ‘new avenues of thought’. 

Some answers show that the programmes can result in a very concrete impact. We 
recognise the transfer to the business environment in the following quotes, through which we 
can see the application of strategy skills as well. 

‘‘The programme offered me to rethink the Dodge 105 brand and vision. It enabled 
me to reorganise our dealer network in order to make it more responsive. And it 



105 Dodge is the name of a Chrysler car brand in the USA. 
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initiated me to review the closed return of the enterprise in terms of efficiency and 
effectiveness in the weeks between the two modules of the programme. ” 

(Participant of the C-Sample) 

“ The programme has helped me very much to prepare the plant Rastatt for the 
future. ... The programme has helped me in the making and the implementation 
of strategy." . 



(Participant of the C-Sample) 

The different answers also show that there has been an application of strategy knowledge as 
seen in the following quotes. 

7 have internalised the corporate decisions and the strategy, and I have realised 
the scope of these topics and understood the dynamics of them as well. ” 

(Participant of the C-Sample) 

“To learn about the strategic challenges and future directions of the company. " 

(Participant of the C-Sample) 

And we can discern changes in strategy mentality as. The next statements of participants 
reflect the aspect of ‘building awareness 

“My perspective has been changed towards a broader and longer-term focused. ” 

(Participant of the C-Sample) 

“Better strategic focus in terms of the competitor analysis in my position. " 

(Participant of the C-Sample) 

Overall, the positive examples of strategy skills are predominately found in the C-Sample. 
Typical responses for the NC-Sample had more a general management background. 

“In all my decisions the new financial knowledge has been considered and 
applied. This has been very good in terms of decisions. Which in turn has been 
good for the company. ” 



(Participant of the NC-Sample) 
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Implementation of new marketing tools for my daily marketing work. " 

(Participant of the NC-Sample) 

Nevertheless, the non-customised programmes also had a focus on strategy. So we found 
successful applications of strategy skills in the responses of the NC-Sample as well. 

“The programme has helped me to make and implement a strategy for my old 
area of responsibility in Scandinavia.” 



(Participant of the NC-Sample) 

“The programme has helped me directly in the making and formulation of 
business plans. ” 



(Participant of the NC-Sample) 

These quotes underline the results presented in table 5.23. Both samples reflect the learning 
on strategy. Yet, the responses from the customised programmes show a higher intensity 
and preciseness in terms of strategy process capability. These qualitative findings support 
the higher ratings with respect to strategy process capability found in the C-Sample in figure 
5.13. 

5.1. 2.4.2 Organisational Outcomes 

The second part of the business perspective consists of a more general assessment of the 
impact of executive education on the organisation. We take a look at the projects initiated, 
the wider aspect of networking and the retention rate. We close the presentation of our 
results with the responses to the question ‘How did the programme create value?’. 

One of the most common forms of organisational change is project work. We have analysed 
the number of new projects initiated by the programmes in order find an indication for the 
degree of action resulting from the programmes. 

In figure 5.17 (on the next page) we find that the outcome of the non-customised 
programmes is more action oriented compared to the customised programme. Over half of 
the participants of the NC-Sample (and the trimmed NC-Sample) have implemented a 
business project as a result of the programme compared to only a third of the members of 
the C-Sample. 
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Figure 5.17: Projects Initiated as a Result of the Programme 

Source: STEER - Follow-up Telephone Interviews 



The explanation for the results in figure 5.17 could be found in the general motivation of 
participating in the programme. Most of the non-customised programmes have been self- 
initiated. Therefore, the willingness to implement the learning could be higher than in the 
customised programmes, who rely predominately on nomination. From the following quotes 
we can see the kind of projects initiated. 

“I have completed successfully a project in the logistics of spare parts. There was 
a clear financial improvement, supported by the new knowledge in finance and 
controlling. ” 

(Participant of the C-Sample) 

7 created a concrete project - the virtual trade exchange. The idea and the 
contacts were created at Harvard and also developed in part there . " 

(Participant of the C-Sample) 

“I have used that advice and insights [from the programme] for the incentive 
package at DaimlerChrysler. In dealing with the issue I have remembered very 
often what the professor has told me and that has helped me a lot in discussions 
and decisions. ” 

(Participant of the NC-Sample) 
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We can see that executive education programmes have the potential to initiate important 
changes in the organisation, which can have a direct financial impact. Yet, it is difficult to 
measure the monetary value of a new idea or an advice. 

We come to another business benefit of executive programmes - the networking aspect. 
One of the most tangible benefits of executive education is the interpersonal integration of 
the organisation. Informal networks have gained in importance in addition to traditional 
organisational structures (Bartlett & Ghoshal 1989). Management development programmes 
support the construction of informal networks within the company, and in the case of the 
open programmes, across companies and industries. 



C-SAMPLE 




How many fellow participants have you contacted since 
the programme? 




21 and 16 to 20 11 to 15 6 to 10 1 to 5 0 NO 

above ANSWER 

Trimmed (o%) (0%) (4%) (22%) (48%) (26%) (0%) 



Figure 5.18: Degree of Networking between Participants 

Source: STEER - Follow-up Telephone Interviews 
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We thus investigated in the potential of executive programmes to build informal networks. 
In figure 5.18 (on the previous page) we find responses on the degree of networking of the 
two types of programmes. The data represents the number of fellow participants that 
participants have contacted in the aftermath of the programmes. The overall distributions for 
the responses of the two programme types look similar. The majority of both samples has 
contacted between one and five fellow participants. Taking a closer look however, we can 
see distinctive differences. 

In the C-Sample almost everybody has contacted a fellow participant for networking 
purposes after the programme (only one percent did not). However, one in five of the NC- 
Sample has not networked at all (21 percent). This figure rises to one in four in the case of 
the trimmed NC-Sample (26 percent). The average number of fellow participants contacted 
reflects the differences. On average participants in the customised programmes have had 
contact with six to seven fellow participants in the aftermath of the programme (average 
6.31). In the NC-Sample only four to five colleagues were contacted (average 4.88). Omitting 
the longer programmes in the NC-Sample, the average of the trimmed NC-Sample is 3.57. 
The difference between the C-Sample and the NC-Sample is almost significant 
(significance level a C to nc = 0.126). The difference between the C-Sample and the trimmed 
NC-Sample is significant at the one percent level (significance level a c to trimmed nc = 0.01). 

We have investigated further and asked for the number of contacts made for professional 
reasons. Thereby, we distinguished between informal networking for no particular reason 
and specific professional networking. Professional networking implies that a former 
participant contacts a fellow participant for a concrete business reason, like the exchange of 
information or the creation of new customer/supplier relationships. In figure 5.19 (on the next 
page) we can see that professional networking is a subset of general networking. 

Professional networking occurs less often. With respect to customised programmes, 
participants use the event to follow-up on business reasons as well. On average three to four 
colleagues are contacted for a specific professional reason. The responses of the NC- 
Sample show a lower average of only two to three contacts. The trimmed NC-Sample is even 
lower at one to two contacts. The customised programmes therefore create almost three 
times as many business contacts as the non-customised programmes. The differences are 
again significant (significance levels actoNc = 0.074 and a c to trimmed nc = 0.007). 
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C-SAMPLE 





Figure 5.19: Degree of Professional Networking 

Source: STEER - Follow-up Telephone Interviews 

Figure 5.19 demonstrates the value of customised programmes with respect to corporate 
integration and knowledge exchange. Executive education can support the creation of 
informal networks and support formal hierarchies. 

We find that non-customised programmes create few professional contacts. Yet, these 
contacts are of an external nature. Depending on the function of an executive, these contacts 
might be very valuable as well. However, the creation of these external ties is more difficult to 
channel by an organisation. 
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In terms of the benefits of networks, the two following quotes show the business relevance of 
informal contacts. 

7 have used the newly gained network within the company and started a 
business project for Diebold Consulting 106 - and that project had very tangible 
results." 



(Participant of the C-Sample) 

“I have created a networking tie to a fellow participant and initiated a very active 
exchange in the topic of supply chain management The person has visited me 
with his team and he took a look at the way we do things here. I am planning to 
visit him with my team soon. " 



(Participant of the C-Sample) 

As we can see above, the customised programmes achieve a higher degree of networking 
activity than the non-customised programmes. Two participants of the NC-Sample describe 
the situation on the external programme and thus give reasons for the lower networking 
activity. 

“We were together at the programme, but after the programme we all departed 
again and there were no more contacts. So I was only able to benefit from other 
participants during the programme itself. ” 



(Participant of the NC-Sample) 

“The diversity of companies made it difficult to apply the learning from them after 
the programme. The participants were very different individuals and therefore 
there was little opportunity to network. It was difficult to work with them after the 
programme. " 



(Participant of the NC-Sample) 

These results show that there is an importance difference in the type and intensity of 
networks, which are created by the respective programme types. Closed, customised 
programmes facilitate intra-company networking. The programmes in the C-Sample have 



106 At the time of the programme, Diebold Consulting was still part of the DaimlerChrysler Group as an internal 
consulting firm. 
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created a significant amount of new personal and professional contacts. Therefore, sending 
an executive to an open programme is a foregone opportunity to build new internal network 
ties. Yet, external programmes offer inter-company and cross-industry contacts. Although 
the networking activity is substantially lower, these contacts might be useful as well. For 
example, high level executives often have to understand future trends in order to anticipate 
and analyse the impact of their strategic decisions. An external network might be beneficial 
to collect the relevant information globally and independently form one’s industry. 

We come to another indicator in the framework of the business perspective - the retention 
rate of participants trained and developed. We acknowledge that the retention rate of 
executives in the company does not depend solely on the amount of executive education. 
However, we consider it useful to include such a broad indicator in the evaluation puzzle. 



Sample 


Leaves Per Sample 

(Between 1 998 and 2000) 


Retention Rate 


C-Sample 


2 (out of 80) 


97.5% 


NC-Sample 


2 (out of 19) 


89.5% 



Table 5.24: Retention Rate of Former Participants in the Organisation 

Source: STEER - Administrative Data from DCU and Business Schools 



In figure 5.24 we observe that a higher share of participants from the C-Sample are still 
working for DaimlerChrysler. Within the period of two years less than five percent of the C- 
Sample left the company. At the same time over ten percent of the NC-Sample left 
DaimlerChrysler (or the respective subsidiaries). We removed those participants from the 
NC-Sample, who followed a programme more than two years ago in order to make the two 
sample compatible. 107 Unfortunately, we were unable to find the corporate-wide retention rate 
as a point of reference. But from own experience we can say that the corporate-wide rate lies 
below the one of the C-Sample (i.e. less retention). 

As mentioned above, there might be various reasons for the differences in the two retention 
rates. Possible reasons could be the shared experience of the programme with colleagues 
from all over the company, the first-hand corporate information and the opportunity for 
identification with the organisation. But of course there might be external reasons causing 
the difference. The following quote however, indicates that executive education might have 
an influence on the retention of participants. 



107 Given a turnover rate of five percent for the NC-Sample, many more participants would have left due to the 
longer time period since the programme (cf. figure 5.6). 
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“The programme has a signalling effect: The company has invested in me, so 
that I am up-to-date and that I can face new challenges.” 



(Participant of the C-Sample) 

The indirect aspects of motivation and identification can also be found in several quotes like 
this one. 

[ The value/purpose of the programme has been ...] “to recharge the battery, to 
refill my motivation, to build my company network, and to strengthen my ties and 
linkage to DaimlerChrysler through a confirmation, recognition and identification. ” 

(Participant of the C-Sample) 

The underlying logic is that organisations do not only reward their employees through pure 
monetary means. Most corporations have an incentive package. Premium executive 
education, like at IMD, INSEAD or Harvard, contribute to the overall attractiveness of the 
employer. Management development therefore creates value in the wider incentive 
framework for executives as well. Hence, we discern that executive programmes do not only 
address learning, but cover additional organisational aspects. 



The last measure of the business perspective is of a qualitative nature again. We have asked 
the participants themselves, how the programme has created value in their view? We find 
the responses in table 5.25. 



C-Sample 


n=72 


Ranking 


“How did the programme create value?” 


Replies 


Percentage 


1 




34 


iHQI 


2 








3 


Learning interculturally and building global awareness 


16 


22% 


4 


Understanding the cultural differences between the different parts of the company 


15 


21% 


5 


Being taught by world-class professors - in terms of their knowledge, experience 
and their teaching skills 


13 


18% 


6 


Understanding our business activities and dynamics 


12 


17% 


7 


Gaining new professional contacts from other business units 


10 




8 


Benefiting from the exchange of new thoughts and ideas 


9 


■eeb 


9 


Learning through the interactive discussions 


9 


sees 


10 




9 


■ESSS 




Valuing the broad selection and composition of the participant group with respect 
to their functional and cultural background 


9 


13% 



(continued on the next page) 
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NC-Sample 


n=34 


Ranking 


“How did the programme create value?” 


Replies 


Percentage 


1 


Gaining relevant and concrete knowledge for my work 


14 


41% 


2 


Building cultural and global awareness 


8 


■Ezm 


3 


Developing general management skills 


8 




4 


Exchanging personal and professional experiences with fellow participants 


8 


■E£^H 


5 




8 




6 




7 


■IBS 


7 


Improving my strategic thinking 


6 




8 


Networking - creating and consolidating business contacts 


6 


18% 


9 


Being taught by world-class professors - in terms of their knowledge, experience 
and their teaching skills 


5 


15% 


10 


Broadening my horizons 


5 


15% 




Refreshing existing skills and knowledge 


5 


15% 




Understanding foreign business cultures and customs 


5 


15% 



Table 5.25: The Value of the Programme as Seen by Participants 

Source: STEER - Follow-up Telephone Interviews 



The responses of the C-Sample confirm the importance of networking in the case of the 
customised programmes. ‘Networking’ is the main value driver in the eyes of the DCU 
participants (47 percent). The emphasis on ‘strategy skills’ can be found in the responses as 
well. Furthermore, the idea of learning more about the company and sharing knowledge 
across the organisation is included in several responses. 

The answers from the NC-Sample reveal the underlying nature of the non-customised 
programmes again. These programmes are more focused on the work of the participants (41 
percent). The focus on the functional responsibilities 108 of participants is in line with the 
results of figure 5.1 1 , which showed a higher rating of relevance for the job in the case of the 
NC-Sample. The motivation to benchmark the knowledge, practices and ideas of other firms 
becomes apparent as well. Participants from the NC-Sample also value the external 
experience in terms of new cultures and new business perspectives. 

Summarising the business perspective - we have presented the financial indicators 
generated by STEER. We have seen that the customised programmes have a slight variable 
cost advantage. Furthermore, we have discovered that, on average, participants of the C- 
Sample value the worth of the programme higher as their real prices. 



108 Functional responsibilities imply the professional field of participants, as for example marketing or research & 
development. 
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The organisational outcomes have confirmed the importance of customised executive 
education for the integration of the company with respect to general and professional 
networking. The retention rate has given us an indication of the signalling effect of closed, 
customised programmes. However, we have seen that non-customised programmes - 
through their greater focus on the functional responsibilities of participants - are more action 
oriented. 

The business perspective provides us with further evidence against the null hypothesis. We 
proved several of the differences discovered as significant. In terms of our puzzle, we are 
confident that we found enough puzzle pieces that indicate that customised and non- 
customised programmes are different. 



5.1.3 Interpretation of the Results from the Comparative Study 

The presentation of results from the four perspectives has provided us with a manifold 
collection of pieces in the evaluation puzzle. In the following section we interpret these 
results with respect to each perspective and subsequently on a general level. 

The main message of the stakeholder perspective is that participants from both samples are 
satisfied with the programme that they have visited. The personal value assessment and the 
comparison to other programmes are similarly positive. The majority of participants stated 
that their expectations had been fulfilled. The uniformity of these positive results can be 
explained by the fact that in both samples the participants’ programme experiences must 
have been similar due to the fact that the customised as well as the non-customised 
programmes are taught by the same business schools (i.e. many variables in the learning 
experiences can be assumed identical). Nevertheless, the stakeholder perspective indicates 
that there is still room for improvement for the customised programmes. The call back rate , 
the cancellation rate and the attendance rate of sponsors indicate that some aspects could 
still be enhanced. 

The content perspective enables us to see the impact of the executive programmes more 
differentiated. We found important differences between the two programme types. First of all, 
the common opinion is confirmed by the results of STEER. Customised programmes offer 
more relevant content with respect to the company, whereas non-customised programmes 
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address more specifically the work and the responsibilities of the participants due to the 
variety of functionally oriented programmes on the open market. We can conclude that 
closed programmes do indeed offer customised content with respect to the company. 

In terms of the specific research angle of our thesis, the content perspective illustrates 
another important difference. Several measures demonstrate that strategy process capability 
is developed more effectively through the customised programmes. The aspect of strategic 
thinking is rated higher in the C-Sample in terms of the five LEAD performance indicators. 
Furthermore, the corporate goals are integrated better in the customised programmes 
(especially when we respect the trimmed NC-Sample). The responses of the three open 
questions on skills, knowledge and mentality provide similar results. We can thus infer that 
the customised programmes have a greater potential to develop the strategy process 
capability than the non-customised programmes. An important reason for this phenomenon 
could be that the focus on one’s own business enables a more tangible learning with respect 
to strategy, i.e. that the examples used during teaching are more concrete and that the 
strategic rationale more visible. 

From the results of the process perspective we can comprehend the rationale of STEER to 
include different perspectives in the evaluation tool. The stakeholder and content 
perspectives have suggested, that the customised programmes are more adequate than the 
non-customised programmes with respect to building strategy process capability. The 
process perspective emphasises that the programme benefits must also be distributed 
effectively and efficiently to the target group. Surprisingly, the internal programmes cannot 
bring to bear their intra-organisational advantage . 109 The external programmes achieve a 
better target group reach. Furthermore, the non-customised programmes offer more flexibility 
and variety in comparison to the customised programmes. From the results of the process 
perspective we can conclude that the customised programmes are not using their full value 
potential in term of the processes of executive education. The recent creation of the 
corporate university can be one of the reasons for the weaker results of this perspective. The 
administrative departments of the business schools have built their professionalism over the 
last decades. They can benefit from well-tested processes and a good reputation of their 
work. 



109 We mean the advantage of having the corporate information and support functions at hand to organise and 
run the programmes. 
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The results of the business perspective round off the evaluation. We perceive that the 
customised programmes are generally regarded as more beneficial to the organisation. The 
value for DaimlerChrysler , the retention rate and above all the degree of networking are in 
favour of the customised programmes. The cost comparison and the willingness to pay 
introduce further favourable measures for the closed, customised programmes (although 
both indicators should be considered with care, due to the difference in the funding of 
programmes 110 ). In- terms of strategy process capability the responses to the question ‘How 
did the programme create value?’ confirm the findings of the content perspective on the 
learning with respect to strategic thinking. 

However, the business perspective also illustrates an aspect, which shows that the results 
are not purely ‘black and white’. The non-customised programmes achieve a higher action- 
oriented output than the customised ones. More projects result from the external 
programmes. Reasons for this could be the higher personal motivation to learn as well as the 
more focused content with respect to the participants’ functional responsibilities. 

Putting the evaluation puzzle together - we can derive an overall interpretation of the 
evaluation results generated by STEER. The results of the four perspectives show that 
generally there is no one-best-way to design and conduct executive education. The value of 
customisation depends on the specific goals, which one would like to achieve through 
executive development programmes. 

The results of the qualitative findings and especially the outcomes of our statistical analysis 
provide good reasons to reject the null hypothesis. We find that there are significant 
differences between customised and non-customised programmes. 

The non-customised programmes offer a flexible way of addressing work-related learning for 
participants. Most management and leadership skills are covered (see LEAD assessment). 
Specific professional areas can be targeted in functionally oriented programmes. 
Furthermore, the open programmes enable a more global benchmark across different 
companies with respect to best practices, experiences and new ideas. These findings 
therefore serve as a confirmation of pre-existing opinions about open programmes (cf. 
section 2.2.2). 

110 The differences in funding refers to the fact, that the customised programmes are paid by the DCU, whereas 
the non-customised programmes are borne by the business unit/department of the participant. 
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However, the results from the customised programmes present new insights with respect to 
the field of research. Our findings confirm that closed programmes can achieve a higher 
relevance of content in terms of corporate issues. Additionally we found that the customised 
programmes provide a more concrete learning experience with respect to strategy process 
capability. The closed atmosphere of customised programmes enable a better environment 
to discuss corporate strategy. More concrete examples can be used - a sense of 
confidentiality is ensured. Thereby, the customised programmes can focus more specifically 
on the relevant topics with respect to corporate strategy. High ranking, internal speakers can 
mediate strategic messages directly and effectively. 

Finally an interesting finding has been, that the internal processes have (not yet) been as 
effective as the external business schools. In terms of the general evaluation, the value 
contribution can be still increased and achieve a higher impact. 



5.1 .4 Repercussions for the Field of Research 

In the following section, we consider possible practical consequences for the field of 
executive programmes. Subsequently, we discuss some of the broader consequences for 
SHRD. 

We have used the results to produce a decision-support matrix (on the next page). The 
matrix is based on our findings and is aligned to the specificities of the field of research . 111 In 
the decision matrix we revise and structure the most important findings of the research with 
respect to general advantages and disadvantages of customisation. We can find many 
‘common sense’ conceptions, which have been confirmed by our evaluation through STEER. 
However, the research results presents some additional insights, for example the more 
action oriented outcomes of the non-customised programmes. 



in 



Generalisations are to be considered with care. Chapter 6 outlines the transferability of our results. 
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Customisation 


Non-Customisation 




• Relevant content for the company 


• Focused content on the specific functional 




• Effective development of strategy process 


responsibilities of participants 




capability in terms of skills and especially 


• High flexibility in terms of increasing and 




knowledge and mentality 


decreasing capacities 




• Confidentiality of 'strategic issues’ 


• High personal motivation with respect to 


tn 


• Direct strategic alignment through internal 


learning 


m 

(0 


speakers 


• More actionable transfer through projects 


■+-« 

c 

(0 


• Effective personal and professional intra- 


• High reputation and professionalism of 


> 

TJ 


company networking 


business schools 


< 


• Good transfer of cross-functional and 


• Broader programme and content diversity 




company specific knowledge 


• Good benchmarking opportunity for best 




• Strong impact on corporate identification 


practices, experiences and new ideas 




and integration 


from other firms and industries 




• Potential for cost and process economies 


• Outside perspective on the company 




of scale 






• Fixed costs 


• ‘Take-it or leave-it approach’ - no way of 




• Less flexible capacities in the short run 


influencing the content, method or lengths 




• Difficulties to attain cost and process 


of the programme 


(/> 

0) 

o> 


advantages (i.e. capacities and 


• Higher variable costs 


iS 

e 


cancellations) 


• Weak networking effect 


03 

> 


• Need to control for and initiate 


• No continuous quality control in terms of 


■o 

03 


customisation 


content 


</) 

Q 


• Missing diversity in terms of participants 


• Individualised and non-corporate learning 




from other businesses 


experience 






• Less influence on the composition of 






participants and their backgrounds 



Table 5.26: Decision-Support Matrix on Customisation 



Returning to the purpose of the evaluation project, our findings have the following 
repercussions for the executive education programmes at the DCU. 

First of all, customisation is of benefit to the organisation. The evaluation by major 
stakeholders, the content feedback and the business perspective show that value is 
generated by the DCU programmes. Customised programmes have the same quality in 
addressing general management learning than programmes on the open market. In terms of 
technical knowledge, for example marketing, the external programmes might offer a broader 
functional variety. However, the closed programmes provide important learning on corporate 
aspects that the non-customised programmes cannot address. 



243 




Secondly, our evaluation results show that the benefit of customisation can be increased 
with respect to the process of executive education, i.e. organisation and realisation. The 
results of the process perspective show that the DCU has the opportunity to outpace 
business school standards on the grounds of its ‘insider advantage’. Currently, the DCU has 
not yet been able to bring to bear its internal ‘monopoly’ position. However, we are confident 
that with more experience and a growing reputation the DCU is likely to improve its process 
performance. Then, the overall value of the programmes will be increased by including more 
participants in the programmes. 

Thirdly, the results demonstrate that customised programmes do not only create value 
through learning. Closed programmes cover many aspects of organisational development, 
as for example networking. Given the challenges of globalisation, these organisational 
effects of corporate integration should not be underestimated. DaimlerChrysler has 
experienced many corporate changes, for example the merger of Daimler-Benz and 
Chrysler. Our results confirm the importance of customised programmes to link different 
parts of the organisation through the creation of informal networks. 

Finally, we come to the specific focus of our research - the value contribution of customised 
programmes to strategy process capability. We have found that the customised programmes 
in our field have been superior in building strategy process capability. Given our 
understanding of capability, we saw that in particular the teaching of strategy knowledge and 
mentality is supported more effectively by customised programmes. Clearly, the closed 
programmes provide more relevant content with respect to the organisation. Internal 
speakers deliver direct and important input on corporate strategy. In terms of mentality, the 
customised programmes create a sense of confidentiality. The closed group enables a 
quicker getting-to-know-each-other and is more powerful in creating awareness of important 
strategic issues and messages. The entire setting of confidentiality and concrete strategic 
input seems to induce a more effective learning experience on strategy process capability. 



5.2 Dimension Two: Inferences on the Quality of the Tool 

"Evaluation is a good concept based on solid theory. But its practice is not well 
developed . " 



(Lange in Foxon 1989, p. 5) 
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Having interpreted the results in part 5.1, we would like to take a bird’s eye view, in order to 
comment on the feasibility of application and the quality of results of the tool STEER. We 
have already mentioned that we combine the theoretical development of a new evaluation 
tool with a first hand practical application of it. In the framework of our thesis we have been 
able to apply the evaluation tool STEER to the comparison of customised and non- 
customised programmes. In the application process we have been able to collect first hand 
feedback on the tool. This feedback consists of our personal experiences of the tool and 
the opinions of DCU staff. In the following sections we thus provide a brief overview on our 
findings on the general quality of STEER. 



5.2.1 Outcomes on the Feasibility of Application 

One of the results of the Theoretical Frame has been the aspect of feasibility and utilisation 
(Patton 1997). We have therefore considered feasibility of application as an important 
element in the development of the evaluation approach. Having tested STEER in a real-life 
situation, we can provide the following experiences in order to come to a tentative 
judgement. 

First of all, we emphasise the variety of evaluation instruments and measures. STEER did 
not focus on one measure alone, but included many different indicators, which were collected 
through a selection of instruments. The diversity of methods had the advantage that the 
evaluator did not depend on one source of data. As we have seen in our application - due to 
unforeseen reasons - some data sources were not accessible. STEER offered flexibility in 
the collection of data as different instruments provided the measures for the four 
perspectives. STEER was thereby quite robust with respect to unexpected problems. On the 
other hand, the diversity can lead to an overwhelming amount of data, which slows down the 
practical analysis. Our suggestion is therefore to test STEER first and then to slim it down to 
the most effective and meaningful measures, i.e. those measures provide the most useful 
information to the evaluator. Hence, at the end of the day, a workable amount of measures 
remains for each perspective. 

Secondly, we review the general experiences with the different evaluation instruments. 
Many research methods used by STEER are well known in social research, as for example 
interviews and questionnaires. Other instruments are less common, as for example the rating 
code for customisation or the cancellation rate. In applying the different evaluation 
instruments we experienced that traditional primary research methods imply a greater 
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independence - in particular in large organisations. We were able to ‘manage’ the direct 
research methods as for example telephone interviews. In contrast, the collection of 
secondary data included many administrative hurdles, i.e. finding the right person and asking 
him to provide the data. In a dynamic business environment, the sources of secondary data 
can change quickly and unexpectedly. Therefore, it is thus useful to combine methods since 
then the evaluation is less affected by organisational changes. 

Thirdly, we include the degree of resourcefulness of the evaluation. Out of the different 
stakeholders defined by STEER, the participants provided the most concrete and resourceful 
feedback. They represented the true customers of executive education and were given 
particular attention. We experienced that other stakeholders had less to say with respect to 
the quality and value of executive programmes. For example, in the complex business 
environment, it seemed difficult for superiors to assess the impact of the executive education. 
However, as a more general reference point we consider it important to include at least some 
kind of feedback from the broader circle of sponsors. 

Fourthly, we consider the value of the four perspectives. We have used the term 
evaluation puzzle repeatedly. The complex structures of the effects of executive education 
do not allow us to assume one linear input-output relationship. Management development 
has different, often dynamic effects and can generate value at different ends in the 
framework of executive education. The structure of the four perspectives gave us a clearer 
guidance in the evaluation research. They enabled us to keep a comprehensive overview of 
the evaluation procedures and results. We thus considered the four perspectives as clearly 
beneficial to the feasibility of the evaluation. They have focused our work on the important 
aspects of executive education. 

The puzzle analogy describes our evaluation problem quite well. Many different pieces can 
be collected and put together in various ways. Thereby, STEER provides structure to the 
collection of puzzle pieces. Through its perspectives STEER ensures that puzzle pieces are 
gathered for the four important aspects of executive education. Without such a structure 
there could be the danger of collecting only pieces that focus on one aspect of education. 
Kaplan and Norton’s theorem “what you measure is what you get” (1992, p. 71) is relevant 
for the evaluation puzzle as well, i.e. the type of pieces at hand might give different 
impressions on the underlying picture. STEER assists that the puzzle pieces are distributed 
evenly on the overall area of the picture (and not only at one corner). Yet, STEER cannot 
guarantee that all the key pieces are discovered. Depending on the evaluation focus, STEER 
provides the analytical support-structure to look for the most important pieces. The most 
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useful and significant pieces, i.e. measures and indicators, are found through repeated 
evaluation procedures. According to Patton’s idea of utilisation , evaluation practice 
determines the most relevant pieces. The primary purpose of STEER has been to evaluate 
internal programmes (according to the research project at DCU). The methodology of 
STEER was therefore devised on the assumption that the evaluator possesses direct access 
to the executive programmes to be assessed. The nature of the non-customised 
programmes made it difficult to collect all the data in the same way than previewed for the 
customised programmes. As a result some indicators could not serve as a mean of 
comparison. 

Overall, we note that STEER is not a closed and rigid evaluation approach. Within the four 
perspectives, additional measures can be added and collected through the predefined 
methods. Unlike the approaches from finance and controlling, the orientation of STEER can 
be qualitative and quantitative. The mixture of qualitative and quantitative date depends on 
the nature of the training and development and the context of the evaluation. The use of 
perspectives and indicators/measures renders STEER flexible in terms of use, which 
ensures feasibility of application. 



5.2.2 Outcomes on the Quality of Results 

Besides its feasibility, we can now comment on the quality of results of the tool as well. We 
have seen in the Theoretical Frame that feasibility and experimental design contrast each 
other, so that an evaluator has to find a compromise between the two. The compromise has 
been one of the challenges of our research. 

First of all, we depict the scope of results. We tried to construct a broad approach that 
comprises many pieces of the evaluation puzzle. Different insights were generated by 
varying perspectives. In our application, the four perspectives produced a wealth of results. 
We regard these results as sufficient in terms of their scope. In our opinion, the different 
perspectives added to the quality of results. We were able to discern, in which perspective 
the programmes created a high value. At the same time we could identify those perspectives 
in which the value has been mediocre or weak. As a result we can now react in a focused 
way by addressing the problems of a perspective, e.g. by reviewing the processes the DCU 
programmes. 
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Secondly, the broad scope of results effected the clarity of results. We acknowledge, that 
due to the amount of perspectives and results it was difficult to arrive at a clear and 
unanimous judgement. The perspectives arrived at differentiated assessments with respect 
to the value generated. For example, STEER showed that the stakeholders were satisfied, 
but the impact in the process perspective could be better. We do not consider the different 
outcomes of the perspectives as a problem. Training and development is a complex field, 
which enables rarely a clear judgement of X is better than Y\ The assessment depends on 
the point of view taken, i.e. the perspective. We consider that more differentiated results are 
of a higher quality than narrower ones. 

A third aspect concerns the reliability of the tool. We applied the tool, its instruments and 
measures, to two different samples in the framework of our comparison. In both cases the 
tool generated sound results that portrayed the differences of the two samples. In particular, 
the important methods, like the participants’ telephone interviews and the analysis of 
administrative data, demonstrated that the tool possesses the necessary reliability for 
evaluation research. Nevertheless, we agree that asking questions in interviews relies on the 
opinions and perceptions of participants, which can influence the reliability of results (Bortz & 
Doring 1995). Yet, we used several methods, which included measures that did not depend 
on interviews and questionnaires, as for example the cancellation rate. 

Fourthly, we consider the validity of results. We applied STEER to a well defined field of 
research consisting of the executive programmes of DCU and open programmes of three 
business schools. Both programme types shared a common overall population of 
DaimlerChrysler executives. In the research on the differences of the two types, STEER 
used instruments, which enabled a quantitative analysis. The non-parametric analysis (e.g. 
Mann-Whitney U-Test) demonstrated the significance of several of the results. Nevertheless, 
the research area of executive education was complex. We realised that it was impossible to 
establish experimental conditions. Therefore, we designed the four perspectives of STEER 
as a mean of checks-and-balances. We attempted thereby to achieve a high construct 
validity - different evaluation aspects were analysed in relation with each other. For example, 
the aspect of strategic thinking was evaluated through a rating question and several open 
questions (i.e. strengths & weaknesses, learned skills, acquired knowledge and changed 
attitude). Hence, each question and its results validated each other respectively. As a result 
of the construct validity, we consider our results as sufficiently valid. 

The fifth quality aspect concerns the usefulness of results. In line with Patton (1997), we 
perceive quality in terms of the practical usability and utilisation derived from the evaluation. 
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We supplied a structured compendium of a number of results, which tried to combine 
academic research and practical orientation. Therefore, some results are not particularly 
useful for the practitioner, for example the statistical analysis that show the difference 
between the two samples. Overall, we designed STEER to produce relevant and useful data 
for the people responsible for the executive education. The first feedback from the DCU 
staff has been very positive on the tool and its results. Several evaluation instruments have 
already been institutionalised before the general implementation and there is a general 
willingness to continue to use the tool. For example, the end-of-programme questionnaires of 
STEER have been used rigorously for over a year now. Several of STEER’S indicators have 
been applied to all DCU activities (for example expectations fulfilled and relevance of 
content). Unfortunately, it has been impossible to compare STEER to other evaluation 
instruments as no other evaluation tool existed before its implementation. However, we 
received very encouraging feedback from experienced members of DCU staff, who have 
worked with other evaluation approaches previously (before they came to DCU). They 
consider the structure of STEER and its indicators as clearly beneficial to the organisation. 

A critical question will be the scope of evaluation activity in the future. We doubt it that the 
DCU will conduct 106 interviews each year. However, we are certain that a reduced, but 
regular evaluation according to STEER will be done. 

Concluding chapter 5- we have presented the results of the first application of STEER in the 
context of a real-life evaluation. We have considered the dual dimensionality of these results. 
First, the generated results of the comparative study have indicated a partial superiority of 
customised programmes with respect to strategy process capability and corporate 
integration. Secondly, the inferences on the quality of the tool itself have demonstrated the 
advantages of flexibility and differentiation with respect to evaluation. 

In the following sixth chapter, we take a critical overall look at the research and its results. 
We discuss possible limitations of the study and assess carefully the applicability and 
transferability of our results. This is done to place the thesis in the wider framework of global 
evaluation research. 
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6 Limitations of the Research 



“ Evaluation is always situational 



(Patton interviewed by Hinn 1998, p. 1) 

Having discussed the application of STEER and its results, we acknowledge that our 
research could still be improved further. The scope and complexity of our field of research 
have limited our possibilities to investigate the effects of customisation in more depth as well 
as in more breadth. 

In the following chapter we briefly review the methodological shortcomings of our 
approach. Moreover, we discuss the applicability and transferability of our results towards 
other research projects. The purpose of this chapter is therefore to inform the reader on the 
limitations of our research. Thereby, we aim to assist the reader in understanding the 
repercussions of our research better and prevent hasty generalisations of our findings. 



6.1 Methodical Shortcomings 

“ Objective data is difficult to provide in any meaningful way. Bottom line 
evaluations may take years to emerge and are unlikely to reflect the true pay- 
back on investment [i.e. ROI]. ” 



(Smith 1993, p. 22) 

Almost any researcher who engages in field work faces the complexity of the real world. As a 
response to the prevailing complexity, he/she thus devises a methodical approach, which 
he/she considers best to investigate the phenomenon in the field. In the framework of our 
thesis we have developed a new evaluation approach, which we consider best to handle the 
complexities of evaluating executive education. We have thus made deliberate choices with 
respect to our research instruments and procedures by weighing the advantages and 
disadvantages. The previous chapters have outlined the benefits of applying STEER. In the 
following part we would like to share the disadvantages of STEER with our reader. 

First of all, our research, like most evaluation studies, faces the same difficulties of 
establishing an exact understanding of the causalities of executive learning (cf. Kammel 
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2000). As mentioned before, learning comprises very complex structures of causes and 
effects. The outcomes of training and development depend on many individual, situational 
and organisational factors. Therefore, we cannot assume a linear causality in terms of input 
and output. We respond to the causality problem by evaluating through four perspectives in 
order to structure and reduce complexity. However, we acknowledge that we are still dealing 
with an evaluation puzzle - attempting to put together an overall picture on the effects of 
executive development. The disadvantage of a puzzle is that the picture is not fully 
complete until all pieces have been fitted together. Our research attempts to find all the 
important pieces, but we cannot be completely sure that we capture all the effects that might 
exist. 

Secondly, the time frame of our research prevented us from conducting a longitudinal study. 
We carried out a comparative study that considered short- and medium-term effects 112 of 
executive education. The short existence of the DCU prevented us from investigating the 
longer-term consequences of its management development programmes (over 24 months). 
We accepted the limited time horizon since we consider the first months after the 
programme to be the most important in terms of the transfer of learning. In the first months 
after the participation, the learning from the programme and the motivation to do something 
are still ‘fresh’. However, we realise that certain organisational effects (cf. Business 
perspective) might take a lot of time to develop fully. A longitudinal study over several years 
would have given us more opportunities to investigate the longer-term impact of corporate- 
wide executive development programmes. 

In terms of the delayed effects, another factor should be considered. The DCU programmes 
have only reached a fraction of their target group so far (overall ca. 15 percent 113 ). Given the 
total target group of ca. 7,000 executives, we cannot be sure whether the executive 
programmes have reached a ‘critical mass’ yet, i.e. whether the organisational effects are 
already noticeable. For example, the full impact of the networking effects will still increase as 
more executives from the target group are reached. 114 The problem of the critical mass leads 
on to our next limitation. 



We define short- and medium-term effects as those changes that occur in the first 24 months after the 
participation in the programme. 

13 This percentage is higher for specific sub-groups in the target population. For example the smaller sub-group 
of Vice Presidents has been reached more broadly. 

4 For example, several former participants stated that their participation in a programme helped them to contact 
other alumni, whom they had not met before. Clearly, the more alumni there are, the more common ground for 
identification exists. 
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Thirdly, our research centres on building strategy process capability, which creates a general 
research problem. In our research, we identified those skills, knowledge and 
mentality/attitudes, that former participants stated they learned and applied in practice. We 
captured those strategy skills, knowledge and mentality with the help of interviews and 
questionnaires. Thereby, we evaluated the contribution of executive education on strategy 
process capability. However, an ideal research design would include the attempt to measure 
the overall impact of teaching strategy process capability as well, i.e. assess to what extent 
developing strategy process capability affects the effectiveness of corporate strategy. The 
ideal research design would then monitor the strategy process of the organisation over years 
in order to be able to discern possible effects. This implies that all intervening variables 
should be controlled in order to isolate the effect of executive education. We are unsure, 
whether such a research design can be realised. The size and complexity of a multinational 
corporation like DaimlerChrysler make these research designs impossible in our opinion. 
However, smaller companies in more stable environments could provide a more appropriate 
environment for such long-term evaluations. 

Fourthly, we would have preferred a larger NC-Population for our research. At the 
beginning of the study, we expected the NC-Population to be larger - considering the size of 
the Total Population of 7,000 executives. However, our research found that the NC- 
Population consisted of only 41 executives. We have thus spent a lot of effort on achieving a 
high response rate with respect to the NC-Sample. After all, we interviewed 34 executives, 
i.e. 83 percent of the NC-Population, who participated in external programmes. 115 
Furthermore, we trimmed the NC-Sample in order to avoid biased responses in terms of the 
differences in programme lengths. As a result, for some questions we could include only 23 
responses in our analysis. Clearly, a larger n would have enabled us to draw more significant 
conclusions from our comparison. 

Finally, we acknowledge that several of our research methods faced the common 
challenges of social research (Bortz & Doring 1995). Our evaluation approach included 
questionnaires and telephone-interviews, which rely on the openness and truthfulness of the 
interviewees. Participants and other stakeholders are the key customers of executive 
education and are thus a vital source of evaluation data. The participants can provide a first- 
hand judgement with respect to the programmes in terms of their value and relevance. Yet, 



115 Getting an interview with the members of the NC-Sample was more difficult than in the case of the C-Sample. 
The executives in the NC-Sample have no connection/affiliation to the DCU, whereas the C-Sample participants 
are DCU alumni with stronger ties to the corporate university. 
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we cannot be completely sure that their answers are not biased by hidden agendas 116 , for 
example Hawthorne effects (Bortz & Doring 1995) might influence the evaluation data. Our 
research relied on asking questions and thus includes perceptions and opinions of people. 
Nevertheless, STEER comprised other research methods as well, which did not rely on 
perceptions. Furthermore, we used the same methods under the same conditions for the two 
samples, so that many effects would occur for both groups and thus ‘cancel out’, i.e. have 
the same influence on both samples and thus do not distort the comparison. 

Overall, we have been aware of these drawbacks of our research. Thus, we designed 
STEER to include many different perspectives and data sources in order to minimise bias. 
After all, our findings (cf. chapter 5) demonstrate that systematic evaluation is possible and 
can generate meaningful results, which can enhance our understanding of the field of 
research. 



6.2 Applicability and Transferability of Results 

"The world of management development is a dynamic one and one that is fraught 
with problems and difficulties, many of which have actually been made worse by 
some of the changes surrounding modern business. ” 

(Paauwe & Williams 2001, p. 103) 

The primary goal of our thesis has been to contribute to the global evaluation research by 
proposing a new evaluation approach. Given our results, we are confident that our thesis can 
shed some light on the complex fields of evaluating executive education and on building 
strategy process capability. However, we are aware of the specificities of our field of 
research, which might limit the applicability and transferability of our results to a more 
general level. In the following section we briefly discuss the four characteristics, which should 
be considered before generalising our findings. 



116 On the one hand, the participants of the customised programmes have been nominated and have participated 
free of charge in the programmes. Considering the political nature of the organisational environment we must 
expect a certain bias. For example, it seems unlikely that participants come up with harsh and devastating 
criticism openly. Some executives are likely to fear, that strong words might reflect badly on them in terms of their 
later careers. On the other hand, participants of the non-customised programmes might want to justify their 
participation in an external programme and not the DCU programme. 
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First of ail, we have conducted our research within one organisation only. Given the 
objective of our research project, we focused our study on evaluating DCU’s executive 
education programmes in comparison to external programmes. Therefore, the scope of our 
research was confined to DaimlerChrysler as an organisation. Given the size and diversity of 
the organisation, we were able to draw from a sufficiently large population of executives (at 
least for the C-Population). Additionally, we were able to hold several organisational 
variables constant, for example one common corporate strategy or one common corporate 
university. However, the focus on DaimlerChrysler implied that we were not able to include a 
broader application of our tool. Many multinational conglomerates certainly accommodate 
similar executive development functions, but new and emerging industries might comprise 
very different HRD practices. As a result, we are unsure whether our results would differ in 
the case of organisations of a smaller size or in different industries. 

Secondly, we recognised that the DCU does not represent a classic corporate university 
in terms of its structure and organisation. As we have seen in chapter 4, the DCU relies on a 
virtual structure, which uses a network of partners to deliver its executive programmes. 
Thereby, the DCU approach differs from the common campus approach of many corporate 
universities in American corporations, as for example General Electric or McDonald’s (cf. 
appendix 8.1). The campus universities possess their own buildings and employ their own 
teaching staff. The nature of the executive programmes might thus be different at the 
campus corporate universities. The aspect of co-operating with external business schools 
might not exist and the aspect of customisation might be of a different nature. Our findings 
may therefore not be transferred due to the different constellation of corporate education. 

Thirdly, the object of our research consisted of the traditional face-to-face programmes, 
which rely predominately on the case study method. The face-to-face format and the case 
study method are still the most common forms of executive education (Garavan et at 
1999b). However, the world of executive education is changing steadily and new forms of 
corporate learning might gain in importance soon. Our findings have therefore some 
relevance for the field of business education today, but this might change in the future. New 
forms of learning/teaching technology, like web-based seminars or online courses may 
change executive education as well. In the case of new forms of executive education, the 
evaluation content of STEER should be reconfigured as well. 

Finally, we would like to comment on the cultural scope of our research. Due to the specific 
research setting, our thesis deals foremost with North American and German approaches to 
management development. In terms of our field research, we discerned that the nature of the 
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executive education, that we have investigated, has been strongly influenced by the North 
American business school education and, to some extent, by European contributions to it. 
We do not consider the influence as a disadvantage as the North American model has 
demonstrated a strong influence on business education globally. Nevertheless, there might 
be significant cultural differences in the way that management development is done locally 
(Avery et al. 1999). In that respect, there might be some limitations in the degree to which 
our findings can be transferred to executive training in other regions like Asia or Eastern 
Europe. 

Overall - we estimate that our research has some relevance for academics and practitioners 
in the field of evaluating executive education. Although we have provided the reader with 
several words of caution in terms of generalising our findings, we see many parallels to other 
global companies and their approaches of management development. Therefore, we are 
confident that our research can be of value to the field. 
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7 Conclusions 



“Development is more like farming than manufacturing. ” 

(Gellermann in Bronner & Schroder 1983, p. 8) 

At the end of our thesis we offer a brief review of the results of our research. Thereby, we 
summarise our research findings in part 7.1 and answer the research questions of our 
Theoretical Frame (LSI -5 in part 2.1) and our Comparative Study (CS1-5 in part 4.2.2). 
Furthermore, we recapitulate the development, design and application of STEER. After 
looking back at our results, we venture a look into the future. Having spent two years in 
the field we place our findings into a broader academic perspective in part 7.2. Thereby, we 
consider future developments in executive education and discuss areas of further research. 



7.1 Summary of Findings 

“Good evaluation works much in the way a detective in a crime novel operates - 
discovering information and ‘clues’, forming hunches, searching for new clues, 
forming new hunches, and so on until the ' mystery ’ of the impact and operation of 
HRD is solved. ” 



(Brinkerhoff 1987, p. 212) 

Most of our research questions have been answered ‘on the way’ in the respective chapters. 
Therefore, the objective of our conclusion is to provide a concise overview of all answers. We 
briefly summarise the results of our literature study (LS) research questions and 
subsequently the research questions of our comparative study (CS). 

In the Theoretical Frame, we defined the content base of our research. Developing strategic 
thinking is one of the most frequent objectives in executive education and therefore was one 
of the foci of our research. 117 We defined strategic thinking as strategy process capability 
and filled the term with meaning. Thereby, we distinguished three elements of strategy 
process capability, i.e. strategy skills, strategy knowledge and strategy mentality. In the 



117 LSI : What is meant by strategy process capability in the framework of strategy-making and implementation? 
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framework of strategy-making and implementation, we offered a typology of strategy process 
capability for four generic organisational levels. 

Subsequently, we discussed Strategic Human Resource Development (SHRD) as a field 
of research that aims at linking training and development to corporate strategy . 118 From the 
literature, we developed three roles of SHRD. First of all, SHRD can have an informing role 
to corporate strategy, i.e. communicating the existing capabilities of human resources to 
decision-makers in the strategy process. Secondly, SHRD can take on an aligning role, i.e. 
developing the capabilities needed for the implementation of corporate strategy. And thirdly, 
SHRD can play an advancing role, i.e. developing strategy process capability of executives 
and other members of the organisation. Our research concentrated on the aligning role in the 
framework of executive education. 

Furthermore, we showed that the aspect of customisation represents a very new area of 
academic research with little empirical work 119 . We outlined the recent developments towards 
more customisation in executive education and depicted the debate on customisation in 
practice. We concluded that it difficult to arrive at a judgement about customisation due to 
missing research in the literature. 

Our discussion of the different approaches in evaluating training and development provided 
an answer to the fourth research question . 120 We reviewed 12 evaluation approaches and 
contributions. Thereby, we structured the results of our literature study into Anglo-Saxon and 
German-speaking approaches, and into contributions from the field of psychology/pedagogy 
and finance/controlling. Furthermore, we compared the different approaches with respect to 
their underlying assumptions, their evaluation goals, their processes and methods and their 
results. We found that few cultural differences exist. Most of the differences have arisen from 
the academic origins of the approaches. 

In the discussion of the different approaches we looked at the contributions with respect to 
measuring the return-on-investment of education . 121 In particular, the Anglo-Saxon 
approaches in finance/controlling provide suggestions for measuring the monetary impact of 
training and development. We concluded that the measurement studies might be able to 
calculate the financial benefit of manual and functional training, as for example a training to 



118 LS2: How can Strategic Human Resource Development help to build strategy process capability? 

119 LS3: What does customisation entail in the area of executive education? 

120 LS4: What theoretical and practical approaches exist to evaluate training and development? 

121 LS5: What are specific efforts to assess the impact -the retum-on-education - of executive education and 
management development? 
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improve driving skills. Yet, we inferred that in the case of abstract executive education, the 
determination of the monetary impact is highly complex and thus often impossible. 

Finally, we identified potential problems and limitations of evaluation research. 122 We 
discerned four problem areas that limit evaluation research significantly. First of all, we 
outlined the problem of the unclear link of causes and effects. Secondly, we perceived a lack 
of appropriate evaluation data. The difficulty to quantify effects in monetary terms and the 
distortion through temporal discrepancies represented the third and the fourth limitations. 

Overall, the analysis of the 12 evaluation approaches in the Theoretical Frame confirmed our 
first assumption in our introduction (part 1.3). Existing evaluation research leaves indeed 
room for improvement. 

As a response to the approaches and problems, we developed a new evaluation tool based 
on the ideas of the balanced scorecard and the four levels of Kirkpatrick’s model. The 
Strategic Tool for Evaluating Educational Returns (STEER) stresses the value of 
perspective in evaluation. STEER consists of four evaluation perspectives, i.e. stakeholder, 
content, process and business perspective. Each perspective contributes a different angle to 
executive education and thus enables a differentiated assessment of the value contribution 
of management development programmes. Each of the four perspectives contains a set of 
evaluation measures that are collected through various measurement methods. 

Overall, we perceive evaluation as putting together the pieces of a puzzle. We found that it 
is almost impossible to judge the complete value of education, however, we can collect 
different pieces of evidence that provide an impression of the value created. Thereby, 
STEER serves as a guide for the construction of the puzzle. Its four perspectives provide the 
co-ordinate system for the construction of the puzzle, i.e. it structures areas in which the 
pieces should be collected. The problem with a puzzle is that it takes a long time to start 
and to find the first connecting pieces. The more pieces fit together, the clearer the picture 
emerges and the quicker the completion becomes. STEER cannot guarantee the completion 
of the picture, but it can accelerate the start of the puzzle. Furthermore, its ensures that a 
minimum of pieces are collected for each corner of the puzzle. 

The underlying message of the puzzle analogy is that in evaluation research it might be 
impossible to find all the pieces and to understand the causes and effects fully. Yet, a 



122 LS6: What are current problems and limitation in evaluation research. 
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sufficient number of important pieces at the right places enables already a well-founded 
broad understanding of the picture, i.e. the value of executive education. 



Evaluation Characteristics 


Strategic Tool for Evaluating Educational Returns (STEER) 


Evaluation Focus 


Four perspectives in evaluation: 

• Stakeholder 

• Content 

• Process 

• Business 


Type of Evaluation 


Summative & formative 


Level of Analysis 


Individual and organisational 
Internal and external 


Evaluator 


Led by an evaluation project manager 

Involving several members of the executive education staff and teachers 


Stakeholders 


Participants, superiors, sponsors, subordinates of participants, content 
providers, external experts, corporate university staff 


Evaluation Process 


Definition of evaluation measures per perspective 

Gap Analysis - comparison of the target with the actual value 


Evaluation Cycle/Timeframe 


Repeated measurements before, during, and several times after the executive 
programme 


Evaluation Methods 


Qualitative evaluation methods (interviews, questionnaires, reports) 

Quantitative evaluation methods (analysis of administrative data, estimations, 
ratings, cost measurement, estimations) 


Strengths 


Inclusion of different evaluation perspectives 
Flexibility and feasibility 
Multiple use of methods 


Weaknesses 


Partly based on perceptions (possible bias) 

Complex management of organisational interfaces and data sources 



Table 7.1 : Summary of the STEER as an Evaluation Approach 



In the Comparative Study of our thesis we applied STEER to evaluate the customised 
executive programmes of the DaimlerChrysler Corporate University (DCU). We decided to 
evaluate the DCU programmes and compare them to external non-customised programmes. 
Thereby, we responded to five research questions (CS1-5). 

Given our theoretical framework of strategy process capability , we found that executive 
programmes taught skills and knowledge relevant to the making and implementation of 
strategy. 123 Additionally, we saw that the programmes built awareness for the importance of 



123 CS:1 Do the executive education programmes of the DCU contribute to the strategy process capability of their 
participants? 
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strategy. We concluded that executive education can indeed contribute to the strategy 
process capability of participating executives. 

The results of STEER enabled us to analyse the differences and similarities of the 
customised DCU programmes and the non-customised external programmes. 124 We 
observed that the settings of these programmes were very similar, i.e. business school 
environment, teaching methods, perceived strengths and weaknesses. Likewise, 
participation in the programmes was considered as clearly valuable by participants of both 
samples. 

The two distinctive differences were the type of content and the selection of participants. 
Our results confirmed that the content of customised programmes was more relevant with 
respect to DaimlerChrysler. However, the difference was not as significant as expected. 
Several participants and sponsors demanded a higher degree of customisation. The non- 
customised programmes provided more focused content in terms of the functional 
responsibilities of participants. 

We discovered that an important difference was the nomination/application process of the 
two types of programmes. In the case of the customised programmes, many participants 
were selected by the central HRD function, i.e. the participants played a predominately 
passive role. Paauwe and Williams (2001) call the traditional nomination a push process, 
i.e. the central HRD institution decides to include certain individuals in the executive 
education. In contrast, most executives of the non-customised programmes initiated their 
participation themselves, i.e. the participants played a more active role. Here, Paauwe and 
Williams (2001) use the term pull process, i.e. executive education is demand-driven and 
pulled by the customer. 

We inferred that the difference in nomination/application could influence the outcome of 
executive education. The non-customised programmes proved to be more action oriented in 
their outcomes as they resulted in a higher rate of projects afterwards. Hence, we concluded 
that the outcome of education depends to a significant degree on the personal motivation 
and attitude towards executive programmes. As a consequence, one could reconsider the 
process of nominating participants. Instead of allocating executives to programmes, 
managers could be asked to decide themselves if they want to apply or not. A more active 
involvement could ensure a higher motivation to learn and to transfer the learning afterwards. 



124 CS:2/3 What are similarities/differences of customised and non-customised executive education programmes? 
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Another important difference was the degree of networking achieved by the two 
programme types. The internal nature of the customised programmes facilitated the personal 
and professional networking significantly better than the external programmes. 

In terms of strategic thinking, we found that the customised programmes had a higher 
potential to build strategy process capability. 125 Thereby, we answered our introductory 
hypotheses (cf. part 1.3). Our findings relied on the results of several measures. The 
quantitative participants’ ratings showed a significant difference and the qualitative 
responses confirmed these ratings. The openness and confidentiality to deal with real 
strategic issues of the organisation seemed to render the learning experience more concrete 
and tangible for participants of the customised programmes. The involvement of high level 
representatives of the company and the direct communication of corporate strategy 
facilitated the creation of strategy knowledge and awareness. 

The results of the first four research questions (CS1 - CS4) have therefore provided us with 
sufficient evidence to reject our underlying null hypothesis. Responding to many critics of 
customised programmes, we can state that we found clearly noticeable differences 
between the two programme types. We confirmed that customisation can be of value to 
the organisation, but we also concluded that the aspect of customisation cannot be explained 
by black-and-white statements. A lot of the advantages and the disadvantages of 
customisation depend on the specific objectives of executive education. Thereby, 
customisation is no panacea to improve executive education generally. At the end of chapter 
5, we provided a decision-support matrix that translated our findings to real-life executive 
education. Our matrix showed that customisation represents a choice that should be made 
on the grounds of the objectives and the situation of the executive education in question. 

Through our research, we confirmed many of the existing perceptions on customisation and 
thus created transparency and assurance for future decisions on customisation in executive 
education. 

Finally, we reviewed the quality of STEER as an evaluation tool. 126 We concluded that the 
evaluation with STEER was indeed feasible. The tool was flexible enough for dealing with 
several problems of collecting data. Nevertheless, it generated sound results, which proved 
interesting for our research questions and for the practitioners in the field. We agreed though, 



CS4: Do customised and non-customised programmes differ in their potential to build individual strategy 
process capability. 

26 CS5: Can the value contribution of either type of programme be measured in a valid and reliable way by 
applying STEER? 
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that evaluation in practice involved compromises and limitations, which we reviewed in 
chapter 6. 

Attempting an overall conclusion, we found that executive education does more than create 
value through the teaching and learning of relevant content. We determined additional 
aspects of executive education that benefit an organisation. Our research revealed the 
importance of building identification with the organisation , integrating the organisation and 
signalling appreciation to executives through education. These intangible benefits are difficult 
to measure directly, but they are certain to impact the bottom line. 



7.2 Outlook and Suggestions for Further Research 

“Practitioners need to see evaluation in a broader context than merely a set of 
techniques to be applied. ” 



(Foxon 1989, p. 5) 

Having reached the conclusions of our research, we face the question of the wider 
implications of our findings, i.e. a justified ‘so what?’. Hence, we place our results and 
experiences in a wider perspective to consider the implications for academic research and 
the business field. Subsequently, we close our thesis with our suggestions for further 
research. 

Executive education remains a controversial issue among academics and practitioners. 
Many authors raise their concerns over the short, but expensive programmes for executives 
(Weiss 2000). They often criticise the traditional teaching methods and the off-the-shelf 
content of customised business school programmes (Stiefel 2000). Especially, many 
practitioners are doubtful on the necessity of high level business education. Yet, the demand 
for face-to-face business school education for executives has risen steadily over the last 
decades (Moe et al. 1999). More and more funds are invested in the education of top 
management levels. Hence, we perceive an underlying dichotomy of opinions concerning 
executive education. Throughout our research we have discovered different world views 
concerning the value of management development shaping the assumptions of decision- 
makers in the field. 
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At the one extreme, we found opinions that consider executive education as a perk, i.e. nice 
to have. For example, we experienced these assumptions in the common responses to the 
announcement, that an executive is going to participate in a management development 
programme. These comments range from ‘ enjoy and relax ’ to ‘have a good holiday’. At the 
extreme, the underlying assumption is that executive education is not essential for successful 
management and thus cannot contribute to the bottom line. As a result, for example, the 
training and development function has often been one of the first areas of cost cutting in 
times of corporate recession (Kailer 1996). 

The other end of the continuum consists of academics and managers who view education 
and life-long-learning as central in ensuring management success. The writings of HRM/HRD 
and SHRM/SHRD demonstrate the belief in the importance of investing in the human capital 
of the organisation. In practice, there is a growing number of organisations that communicate 
and live the importance of education, including high level management development (Hackett 
1997). Thereby, several companies demonstrate the central role of executive education 
through a clear financial commitment (Stewart 1991; Densford 1999). 

We gather that the assumed importance of executive education directly influences the need 
and the nature of evaluation. Thereby, the role of evaluation changes according to the 
location on the continuum of opinions on education. For example, given the nice-to-have 
extreme, evaluation is required to prove the value of executive education in general. The 
question to be answered here is, 'Does management education make sense or not?’ 
Thereby, evaluation is seen as a means to justify the investment in education. 
Consequently, a high emphasis is placed on ROI studies. This implies a purely summative 
evaluation approach. 

At the other end of the continuum, the question of ‘Do we need executive education?’ is 
implicitly answered in the underlying assumptions. Academics and practitioners agree on the 
necessity and value of education. Therefore, evaluation plays a different role. Here, the 
question to be answered is, ‘Are we investing in the right executive education?’ Thereby, 
evaluation assists in optimising the value generated by executive programmes. We discern 
a combination of summative and formative evaluation approaches. 

For the future of evaluation, we expect combined evaluation approaches that include 
summative as well as formative objectives to gain in significance. The importance of 
knowledge and learning is likely to increase even further in tomorrow’s business 
environment. Hence, we anticipate that the group at the nice-to-have end is going to become 
smaller over time. We have already seen a change in mentality in companies that have 
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realised the central role of corporate education. 127 We expect that this trend will continue and 
that consequently management education will play a more central part in organisations, 
probably also in the form of customised programmes and events. As a result, we expect the 
role of evaluation to change as well. The isolated attempts to prove the value of a 
programme will be less important. Instead, the more comprehensive and integrated 
approaches will be demanded. Combining summative and formative objectives, evaluation 
will continue to play a central role in the design and realisation of executive education. 
Thereby, the focus of evaluation will not be to prove the value in education, but to ensure that 
value is created most effectively and efficiently. 

Our outlook into the future leads on to the need for further research. Thereby, we 
distinguish between research on specific questions that arose during our research and more 
global research suggestions. 

First of all, during our research we were unable to make full use of the LEAD data from the 
performance appraisal system. However, the implementation of the corporate-wide, 
standardised appraisal system has been completed recently and thus enables a longitudinal 
study into the effects of executive education. Given the growing number of participations and 
the size of the overall population, the appraisal system provides interesting data for a large- 
scale statistical analysis. Thereby, control groups could be defined and the participation in 
the programme could be isolated as the independent variable. It would be interesting to see, 
whether our impression of table 5.7 would be confirmed or rejected. 

Secondly, the impact of self-motivated applications vs. nomination remains to be examined 
further. The more active involvement of executives could be desirable in order to pull 
executive education. A study comparing self-initiated vs. selected participants would be of 
value. It could provide important insights into the value of nomination and the value of self- 
application to programmes. 

Thirdly, we were unable to include all stakeholders in the evaluation as planned. Therefore, 
we propose a more systematic and open involvement of colleagues, subordinates and 
superiors of participants in the feedback process. Given more time and resources these 
stakeholders could be approached on a broader basis in order to capture their experiences 
of the impact from the participation of ‘their’ executives. 



127 



In a way, the proliferation of corporate universities is one of many indicators for this trend. 
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Fourthly, we acknowledge that we studied a particular form of business education, i.e. 
common business school programmes that rely predominately on the case study method. 
However, other forms of management development exists (cf. Garavan et al. 1999b), which 
could provide further dimensions of comparison. We focused on the customisation of 
content, but we encourage research that includes new types of executive education as well. 
For example, action learning (Gibbons 1999) takes a new approach to executive learning, 
which has received considerable attention in the literature. STEER could be used to compare 
class-room education with action learning. 

On a more global level, our findings indicate the following research opportunities. On the 
one hand, we consider the opportunity to network as a central value driver of executive 
education. Yet, to our knowledge, the organisational impact and importance of these informal 
networks remain under-examined. We would welcome research in the use and benefits of 
personal contacts in an organisation. More empirical evidence on the value of networks could 
explain the importance of internal executive education for the integration of the organisation. 

Secondly, we welcome further applications of the tool STEER. The evaluation approach 
should be tested by somebody, who has not been involved in the development of STEER. 
We are certain that thus new clues on the applicability and utility of the tool would be 
generated, which could be used to improve our approach. 

Thirdly, we suggest studying the way other organisations handle the aspect of customisation. 
The nature and scope of the co-operation with external business schools and universities 
might be different. Compared to our findings, a broader set of lessons learned from 
customisation could be established. 

Finally, our research indicated that more evaluation is not always preferred to less. We 
outlined that over-evaluation can be a waste of resources (cf. Patton 1997) and can lead to 
dysfunctional behaviour (cf. Kailer 1996). Therefore, we would find it interesting to determine 
how much evaluation is really needed? There have been few studies on the actual use of 
evaluation data in organisations. We agree that the extent to which evaluation results are 
used should determine the type and intensity of an evaluation approach. Consequently, a 
study that indicated the role that evaluation data plays in decision processes, would be 
beneficial to evaluators and users of evaluation. The findings could help to focus the 
evaluation effort and to avoid ritual-like assessments. Overall, we believe that evaluation 
should not be done only for the evaluation’s sake. 



265 




In the flow of our thesis we have made frequent use of quotes. Therefore, we would like to 
close with a quote as well. 

" You think training is expensive? Try ignorance!" 



(Frietman et al. 2000, p. 113) 
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8.1 The Origins of the Concept of the Corporate University: Two Examples 

The founding idea for the concept of the corporate university stems from the early 1950s and 
originated in the United States. In order to illustrate the beginnings of the concepts it is 
worthwhile to take a brief look at two of the most famous examples for corporate universities. 

The Idea of the In-house Training Institution: The Case of General Electric 

In the early 1950s General Electric underwent a restructuring programme with the aim to 
decentralise its organisation. Ralph Cordiner, the Chairman at the time, realised that the new 
structure would require a new set of skills and a changed mindset among its executives. He 
appointed a group of managers and consultants to investigate possible training and 
education providers. None of the existing business schools and universities however could 
accommodate the requirements of General Electric at the time. As a result, Cordiner decided 
to establish a company-owned training facility which became known as “Crotonville”. The 
idea of the in-house training and development centre was born. Crotonville gained particular 
attention in the early 1980s. The newly appointed Chairman John F. Welch re-defined the 
role and function of Crotonville and made it one of its most important strategic instruments to 
drive General Electric’s organisational transformation in the mid 1980s. It was then, that 
Crotonville developed to be a global benchmark for the concept of corporate universities 
(Deiser 1997). 

The Coining of the Term Corporate University: The Case of McDonald’s 

The idea of using the term “university” to describe a training and development centre 
originated in the early 1960s also in the United States. The rapid expansion of its restaurants 
forced McDonald’s to ensure a common standard among its franchisees. The founder of the 
company Ray Kroc saw the need for training and established an institution that he called 
“Hamburger University”. McDonald’s perceives its university primarily as a instrument to 
ensure standardisation within its organisation. The university concepts has developed so far 
that employees are able today to graduate with a “Bachelor of Hamburgerology”. 

The two cases illustrate the shaping of the history of the corporate university concept. Both 
original ideas however were far from what is understood by the concept today. The idea 
evolved over the years and spread within North America (Meister 1998a). Other corporations 
realised the advantages of an in-house training and development institution and established 
their own. The most famous examples include Accenture’s “Centre for Professional 
Education in St. Charles" (est. 1970), “Motorola University" (est. 1981) and “Apple 
University” (est. 1985) (Aubrey 1999). 
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8.2 A General Core Competency Model (Burack et al. 1998) 



All Employees 


-► Mid-Level Managers 


— ► Senior Managers 


Flexible/Adaptive to Change 


Change Implementor 


Change Leadership 


Stress Coping 


Creative 


Persuasive Communicator 


Customer Service-minded 


Strategic Thinking 


Strategic Initiator 


Open-mindedness 


Cultural Diversity/Sensitivity 




Team Player 


Team Builder 




Co-operative 


Participation-oriented 





Source: Burack etal. 1998, p. 19 



8.3 Strategic Thinking at the Individual Level (Bonn 2001) 

Strategic thinking at the individual level comprises three main elements: 

1 . A holistic understanding of the organisation and its environment; 

2. Creativity; and 

3. A vision for the future of the organisation. 

Source: Bonn 2001 , p. 64 



8.4 The Elements of Strategy Thinking (Liedtka 1998b) 



Systems 

Perspective 



Intent 

Focused 



Strategic 




Source: Liedtka 1998b, p. 122 
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8.5 Description of Strategic Thinking (Fauth 1992) 



Definition of the Characteristics for the Job Description of Executives 

(1) Professional Expertise 

• Know-how, knowledge and experience in the respective professional field 

• Broad knowledge of neighbouring professional fields 

• Talent for organisation 

• Marketing and sales orientation 

• Experiences abroad (in a foreign county) 

• Language skills 

(2) Personal Competence 

• Communication and Co-operation skills 

• Delegation behaviour 

• Creativity 

• Stability and self-confidence 

• Ability to work in teams 

• Ability to integrate 

• Business acumen 

• Resistance 

(3) Methodical Competence 

• Decision making 

• Analytical thinking and acting 

• Setting objectives, planning and evaluation of success 

• Information management 

• Individual and group work skills 

• Organisation development 

(4) Entrepreneurial Competence 

• Achieving corporate goals and success 

• Achieving productivity 

• Innovation, vision 

• Strategic thinking and planning 

- Does the manager analyse his/her future tasks clearly and precisely? 

- How is his/her knowledge of the market and the competitors? 

- Can he/she formulate corporate goals comprehensively, realistically and in line with the market? 

- How successful is he/she in creating and developing competitive advantage? 

- Does he/she consider current trends and possible decisions in the future and is he/she able to 
question his/her actions at any time? 

- Does he/she possess sufficient methodical competence? 

- Are strategic planning and operational work in a reasonable proportion to each other? 

- Are strategic goals realised consequently in practice? 

• Integrated thinking 

• Employee development 

• External engagement 

Source: Fauth 1992, pp. 230-235 
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8.6 Description of Strategic Thinking (Larreche 1999) 



“An individual's ability to make strategically sound decisions involves several 
skills: 

• To be able to quickly grasp the most important elements of situations, 

• To integrate the impact of possible competitive actions, 

• And to commit to a course of action while maintaining alternative contingency 
plans . " 

(Larreche 1999, p. 1) 



8.7 Characteristics of Strategic Human Resource Development 



SHRD Characteristics 


SHRD Characteristics 


(Garavan 1991, pp. 17-23) 


(McCracken & Wallace 2000, p. 287) 


(1) Integration with organisational missions and goals 


(1) Shaping organisational missions and goals 


(2) Top Management support 


(2) Top Management leadership 


(3) Environmental scanning 


(3) Environmental scanning by senior management, 
specifically in HRD terms 


(4) HRD plans and policies 


(4) HRD strategies, policies and plans 


(5) Line manager commitment and involvement 


(5) Strategic partnerships with line management 


(6) Existence of complementary HRM activities 


(6) Strategic partnerships with HRM 


(7) Expanded trainer role 


(7) Trainers as organisational change consultants 


(8) Recognition of culture 


(8) Ability to influence corporate culture 


(9) Emphasis on evaluation 


(9) Emphasis on cost-effectiveness evaluation 



Source: McCracken & Wallace 2000, p. 287 
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8.8 Bronner and Schroder’s Control Group Design 



Number of Measurements 


Number and Nature of Groups 


Control-Design 

(Sequence C, T, P) 


1 


1 to n groups 




T 


C 


2 


1 to n groups 


Ci 


T 


c 2 


4 


2 groups 
Training group 


Ci 


T 


c 2 




Control group 


Ci 




c 2 


6 


3 groups 
Training group 


Ci 


T 


c 2 




Control group 1 


Ci 




c 2 




Control group 2 


Ci 


P 


c 2 


5 


3 groups 
Training group 


Ci 


T 


c 2 




Control group 1 


Cl 




c 2 




Control group 2 


0 Cl 


T 


c 2 


7 


4 groups 
Training group 


Cl 


T 


c 2 




Control group 1 


Ci 


P 


c 2 




Control group 2 


Cl 




c 2 




Control group 3 






c 2 


C = Measurement of specific criteria 

T = Training intervention 

P = Placebo training intervention 

0 = Average 



Source: Bronner & SchrSder 1983, pp. 57-58 
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8.9 The Programme Evaluation Standards 



Summary of Standards 

The Joint Committee on Standards for Educational Evaluation 



Utility Standards 



The utility standards are intended to ensure that an evaluation will serve the information needs of 

intended users. 

U1 Stakeholder Identification Persons involved in or affected by the evaluation should be 
identified, so that their needs can be addressed. 

U2 Evaluator Credibility The persons conducting the evaluation should be both trustworthy and 
competent to perform the evaluation, so that the evaluation findings achieve maximum 
credibility and acceptance. 

U3 Information Scope and Selection Information collected should be broadly selected to address 
pertinent questions about the programme and be responsive to the needs and interests of 
clients and other specified stakeholders. 

U4 Values Identification The perspectives, procedures, and rationale used to interpret the findings 
should be carefully described, so that the bases for value judgements are clear. 

U5 Report Clarity Evaluation reports should clearly describe the programme being evaluated, 
including its context, and the purposes, procedures, and findings of the evaluation, so that 
essential information is provided and easily understood. 

U6 Report Timeliness and Dissemination Significant interim findings and evaluation reports 
should be disseminated to intended users, so that they can be used in a timely fashion. 

1)7 Evaluation Impact Evaluations should be planned, conducted, and reported in ways that 
encourage follow-through by stakeholders, so that the likelihood that the evaluation will be used 
is increased. 
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Feasibility Standards 



The feasibility standards are intended to ensure that an evaluation will be realistic, prudent, diplomatic, 

and frugal. 

FI Practical Procedures The evaluation procedures should be practical, to keep disruption to a 
minimum while needed information is obtained. 

F2 Political Viability The evaluation should be planned and conducted with anticipation of the 
different positions of various interest groups, so that their co-operation maybe obtained, and so 
that possible attempts by any of these groups to curtail evaluation operations or to bias or 
misapply the results can be averted or counteracted. 

F3 Cost Efficiency The evaluation should be efficient and produce information of sufficient value, 
so that the resources expended can be justified. 



Propriety Standards 



The propriety standards are intended to ensure that an evaluation will be conducted legally, ethically, 

and with due regard for the welfare of those involved in the evaluation, as well as those affected by its 

results. 

PI Service Orientation Evaluations should be designed to assist organisations to address and 
effectively serve the needs of the full range of targeted participants. 

P2 Formal Agreements Obligations of the formal parties to an evaluation (what is to be done, 
how, by whom, when) should be agreed to in writing, so that these parties are obligated to 
adhere to all conditions of the agreement or formally to renegotiate it. 

P3 Rights of Human Subjects Evaluations should be designed and conducted to respect and 
protect the rights and welfare of human subjects. 

P4 Human Interactions Evaluators should respect human dignity and worth in their interactions 
with other persons associated with an evaluation, so that participants are not threatened or 
harmed. 

P5 Complete and Fair Assessment The evaluation should be complete and fair in its examination 
and recording of strengths and weaknesses of the programme being evaluated, so that 
strengths can be built upon and problem areas can be addressed. 

P6 Disclosure of Findings The formal parties to an evaluation should ensure that the full set of 
evaluation findings along with pertinent limitations are made accessible to the persons affected 
by the evaluation, and any others with expressed legal rights to receive the results. 

P7 Conflict of Interest Conflict of interest should be dealt with openly and honestly, so that it does 
not compromise the evaluators processes and results. 

P8 Fiscal Responsibility The evaluator’s allocation and expenditure of resources should reflect 
sound accountability procedures and otherwise be prudent and ethically responsible, so that 
expenditures are accounted for and appropriate. 
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Accuracy Standards 



The accuracy standards are intended to ensure that an evaluation will reveal and convey technically 

adequate information about the features that determine worth or merit of the programme being 

evaluated. 

A1 Programme Documentation The programme being evaluated should be described and 
documented clearly and accurately, so that the programme is clearly identified. 

A2 Context Analysis The context in which the programme exists should be examined in enough 
detail, so that its likely influences on the programme can be identified. 

A3 Described Purposes and Procedures The purposes and procedures of the evaluation should 
be monitored and described in enough detail, so that they can be identified and assessed. 

A4 Defensible Information Sources The sources of information used in a programme evaluation 
should be described in enough detail, so that the adequacy of the information can be assessed. 

A5 Valid Information The information gathering procedures should be chosen or developed and 
then implemented so that they will assure that the interpretation arrived at is valid for the 
intended use. 

A6 Reliable Information The information gathering procedures should be chosen or developed 
and then implemented so that they will assure that the interpretation obtained is sufficiently 
reliable for the intended use. 

A7 Systematic Information The information collected, processed, and reported in an evaluation 
should be systematically reviewed and any errors found should be corrected. 

A8 Analysis of Quantitative Information Quantitative information in an evaluation should be 
appropriately and systematically analysed so that evaluation questions are effectively 
answered. 

A9 Analysis of Qualitative Information Qualitative information in an evaluation should be 
appropriately and systematically analysed so that evaluation questions are effectively 
answered. 

A10 Justified Conclusions The conclusions reached in an evaluation should be explicitly justified, 
so that stakeholders can assess them. 

All Impartial Reporting Reporting should procedures should guard against distortions caused by 
personal feeling and biases of any party to the evaluation, so that evaluation reports fairly reflect 
the evaluation findings. 

A12 Metaevaluation The evaluation itself should be formatively and summatively evaluated against 
these and other pertinent standards, so that its conduct is appropriately guided and on 
completion, stakeholders can closely examine its strengths and weaknesses. 

Source: Sanders, The Joint Committee on Standards for Educational Evaluation 1994, Inside Cover 
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8.10 All Forms and Questionnaires 



Program Name: 



Standardized Assessment Sheet 

Date: 



Please assess and tick: 



0 



Thank you! 



Your Program - 


Five Key Evaluation Measures 












Expectations Overall 


Duration 




Relevance 




Overall 




Value 








to my job 




Learning 


Exceeded 1 | 


Very High | | 


Perfect 


□ 


Very relevant 
to mv iob 1 1 


1 learned 
a lot 


□ 


□ 


□ 




n 




□ 


□ 


Fulfilled 1 | 


Medium | | 




□ 




□ 




□ 


□ 


□ 




n 




□ 


1 learned 
very little 


□ 


Disappointed! [ 


Very Low | | 


Inadequate 


□ 


Not relevant i 1 

to my job | | 


□ 


Your Personal Learning 














Thinking 


Leading 


Motivating 




Sharing 




Creating 


Strategically Change 


People 




Knowledge 




Value 


1 learned 
a lot 1 1 


1 learned 
a lot 1 1 


1 learned 
a lot 


□ 


1 learned 
a lot 1 1 


1 learned 
a lot 


□ 


□ 


□ 


i i 




□ 


□ 


□ 


□ 




□ 




□ 




□ 


□ 


□ 




□ 




□ 




□ 


1 learned 1 1 

very little | | 


1 learned 1 1 

very little | | 


1 learned 
very little 


□ 


1 learned 1 1 

very little | | 


1 learned 
very little 


□ 


Your Professional Learning 














Delighting 


Increasing 


Managing 


Sustaining 


Integrating 


Going 


Customers 


Profitability 


a Portfolio 


Growth 


Operations 


Global 


1 learned 
a lot 1 1 


1 learned 1 learned 

a lot 1 1 a lot 1 1 


1 learned 
a lot 


□ 


1 learned 

a lot 1 1 


1 learned 
a lot 1 1 


□ 


□ 


□ 


u 


□ 




□ 


□ 


□ 


n 




□ 


□ 




□ 


□ 


□ 


□ 




□ 


□ 




□ 


1 learned i 1 

very little | | 


1 learned i 1 1 learned i 1 

very little | | very little | | 


1 learned 
very little 


□ 


1 learned i 1 

very little | | 


1 learned i 1 

very little | | 



Your Networking 



How many close contacts did you make? Contacts 



Your Evaluation 

How much was this program worth to you? $ 



Your "Key Take-Away" from the Program 



Your Comments 
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Participant Follow-up Interview 




A6 (LEVEL) 

Executive level of participant 

□ ( 3 ) E3/ Band 94-95 

□ ( 2 ) E2/ Band 96 

□ ( 1 ) El/ Band 97 

□ ( C ) Level C 



A7 (PROGRAM) 

Type of Program participated in 

□ (INSEAD) Managing Business 

□ (Harvard) Managing Global Opportunities 

□ (IMD-L) Leadership in a Cross-Cultural Environment 

□ (IMD-GM) Driving Success in a Global Company 



□ ( ) 

A8 (CUSTOMIZATION) 

Program form 

□ ( 2 ) Customized 

□ ( 1 ) Non-customized 

A9 (DATE SEMINAR) (DATE INTERVIEW) 

Date of seminar (start date) Date of interview 

[A9 (3,6,12,18,24)] 
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INTRODUCTION 



B1 (DECISION) 

Who initiated the idea of your participation in the program? 

□ ( 5 ) Nomination by EMD 

□ ( 4 ) Idea of superior(s) 

□ ( 3 ) Advice of colleagues 

□ ( 2 ) Own initiative 

□ ( 1 ) Other: . 

□ (NA) NO ANSWER 

B2 (REASONS) 

What were the objectives for your participation in the program? 
(For example with respect to LEAD) 



B3 (REASONS FULFILLED) 

To what extent were these objectives fulfilled? 



Page 2 
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THE PROGRAM 



Cl (COMPARISON) 

How did the XYZ program compare to 
similar management development 
programs that you have participated in? 

□ ( 5 ) One of the best 

□ ( 4 ) Better than average 

□ ( 3 ) Average 

□ ( 2 ) Worse than average 

□ ( 1 ) One of the worst 

□ (NA) NO ANSWER 

C3 (STRENGTH) 

Looking back - what were the strengths of the program? 
(What did you like about the program?) 



C2 (ENJOYMENT) 

How much did you enjoy the program? 
How much fun was it? 

□ ( 5 ) Completely 

□ ( 4 ) Mostly 

□ ( 3 ) Moderately 

□ (2) Slightly 

□ ( 1 ) Not at all 

□ (NA) NO ANSWER 



C4 (WEAKNESSES) 

What were the weaknesses of the program? 
(What didn't you like about the program?) 



Page 3 
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RELEVANCE 



D1 (UP-TO-DATE) 

Did the program content respond to the challenges of DaimlerChrysler at the time? 



□ 


(5) 


Completely 


□ 


(4) 


Mostly 


□ 


(3) 


Moderately 


□ 


(2) 


Slightly 


□ 


(1 ) 


Not at all 


□ 


NO ANSWER 



D2 (RELEVANCE) 

How relevant was the content of the program to YOUR job? 

□ ( 5 ) Very relevant 

□ ( 4 ) Mostly relevant 

□ ( 3 ) Medium (average) 

□ ( 2 ) Slightly relevant 

□ ( 1 ) Not at all relevant 

□ ( NA ) NO ANSWER 

D3 (APPLICABILITY) 

How much of the program content did you apply directly to your job? 

Please estimate in percent. 

percent. 

D4 (TOOLS) 

How often have you used the tools or insights that were presented during the program? 

□ ( 5 ) Very often 

□ (4) Often 

□ ( 3 ) Moderately 

□ (2) Seldom 

□ ( 1 ) Never 

□ NO ANSWER 

D5 (TOOLS WHICH) 

Please elaborate. Which ones? Or in the case of “No”: Why not? 
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PERSONAL LEARNING 

El (SKILLS) 

Which skills did you learn or improve? 




E2 (KNOWLEDGE) 

What knowledge did you gain or extend? 




E3 (ATTITUDE) 

Did the program change your opinion or attitude? Please elaborate. 






LEARNING WITH RESPECT TO LEAD 



FI (STRATEGY) 

Did the program help you to think more strategically (compared to before the program)? 

□ ( 5 ) Completely 

□ ( 4 ) Mostly 

□ (3) Moderately 

□ (2) Slightly 

□ ( 1 ) Not at all 

□ (NA) NO ANSWER 

F2 (CHANGE) 

Did the program help you to implement change? 

□ ( 5 ) Completely 

□ (4) Mostly 

□ ( 3 ) Moderately 

□ (2) Slightly 

□ ( 1 ) Not at all 

□ (NA) NO ANSWER 

F3 (PEOPLE) 

Did the program help you to lead people? 

□ ( 5 ) Completely 

□ (4) Mostly 

□ ( 3 ) Moderately 

□ (2) Slightly 

□ ( 1 ) Not at all 

□ (NA) NO ANSWER 

F4 (KNOWLEDGE) 

Did the program help you to share knowledge? 

□ ( 5 ) Completely 

□ ( 4 ) Mostly 

□ ( 3 ) Moderately 

□ (2) Slightly 

□ ( 1 ) Not at all 

□ (NA) NO ANSWER 

F5 (VALUE CREATION) 

Did the program help you to create business value for DaimlerChrysler? 

□ ( 5 ) Completely 

□ ( 4 ) Mostly 

□ ( 3 ) Moderately 

□ (2) Slightly 

□ ( 1 ) Not at all 

□ (NA) NO ANSWER 
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PROFESSIONAL LEARNING 



G1- 6 (CORPORATE GOALS) 

Please rate your professional learning with respect to the corporate goals. 







Very 

High 


High 


Medium 


Low 


No 

Value 


No 

Answer 




PROFESSIONAL LEARNING 


(5) 


(4) 


(3) 


(2) 


(D 


(NA) 


G1 


Delighting Customers 














G2 


Increasing Profitability 














G3 


Managing a Portfolio 














G4 


Sustaining Growth 














G5 


Integrating Operations 














G6 


Going Global 















AFTER THE PROGRAM 

HI (DONE DIFFERENTLY) 

What have you done differently as a result of the program? 



H2 (PROJECT) 

Did you start to implement any business project as a result of the program? 

□ ( 2 ) Yes 

□ ( 1 ) No 

□ (NA) NO ANSWER 
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NETWORKING 



II (PARTICIPANTS) 

In how far have you benefited from your fellow participants? 



12 (NETWORKING) 

How many of the fellow participants have you contacted since the program? 

Number of contacted executives: 

13 (NET PURPOSE) 

How many of the participants have you contacted for a specific professional reason? 

Number of contacted executives (professional reason): 

14 (OPEN/CLOSED) 

What would have been different, if the program had been “open” with respect to its 
participants? (I.e. including participants from other companies). 
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VALUE 

Please compare and rate the value of the program for ... 



J1 (VALUE YOU) 
YOU personally. 

□ ( 5 ) Very high 

□ (4) High 

□ ( 3 ) Medium 

□ (2) Low 

□ ( 1 ) No value 

□ (NA) NO ANSWER 

J3 (WORTH) 



J2 (VALUE DC) 
DaimlerChrysler in general. 

□ ( 5 ) Very high 

□ (4) High 

□ (3) Medium 

□ (2) Low 

□ ( 1 ) No value 

□ (NA) NO ANSWER 



Looking back - how much would the DCU program be worth to you in DOLLARS? 

Please state the amount that you are willed to invest in a DCU program similar to your one. For 
example, imagine that you would like to send a colleague or employee of yours to the same DCU 
program. 

For the calculation please consider the entire program - including accommodation, BUT excluding 
travel expenses. 

Perceived Dollar value. $. 



J4 (WORTH CONSTRUCT) 

How did you arrive at that figure? 



J5 (HOW VALUE) 

In conclusion, how did the program create value? 



Thank you very much! 
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Sponsor Interview 



SIX OPEN QUESTIONS 



Q1 (ROLE) 

Please describe your role/ involvement with the DaimlerChrysler Corporate University. 



Q2 (EXPECTATIONS) 

As a “sponsor”/”initiator”/”stakeholder” - what do you expect from the DaimlerChrysler 
Corporate University in terms of its organizational role? 



Q3 (SUCCESS FACTORS) 

What are critical success factors for the DaimlerChrysler Corporate University? 



Q4 (BARRIERS) 

What are barriers to success? 



Q5 (VALUE ADD) 

How do customized management development programs - like INSEAD’s “Managing 
Business” - create value for DaimlerChrysler? 



Q6 (SUCCESS) 

How could you tell - in your role - that a DCU program has been successful? 



Thank You Very Much! 




8.11 Programme Descriptions 



Customised 

Programmes 


Target Group 


Brief Programme Description 


Managing Global 
Opportunities 
2x 1 week 
DCU/ Harvard 
Business School, 
Boston, MA, USA 


Executives Level 1/Band 97 
and selected Level 2/Band 96 


As a DaimlerChrysler top management program, this seminar develops 
leadership and general management skills of top level executives with 
potential to assume a more senior role. 

The two modules (one week each) expand participants’ general 
management skills, supporting them to rethink framework conditions and 
discuss possible solutions for strengthening DaimlerChrysler in global 
markets. The course is aligned to the DaimlerChrysler strategies and key 
initiatives to meet the future challenges involved with DaimlerChrysler*s 
role as a global player. 


Managing Business 
2x 1 week 
DCU/ INSEAD, 
Fontainebleau, 
France 


Level 2/Band 96 executives 
and selected Level 3/Band 
94-95 executives with 
potential to assume a more 
senior role 


This seminar is designed for the senior management to further develop 
general management and leadership capabilities. 

The two-module program develops and expands business administration 
skills and increases the ability to act as an entrepreneur within 
DaimlerChrysler, discussing ways in which they may enhance the 
competitiveness of their business. In the seminar, the participants work on 
problems from their own areas of responsibility. The solutions found can 
be directly implemented in their working environment. The exchange of 
best practice knowledge between the participants and the business school 
lecturers is a central focus of the course. The program will provide the 
concepts, framework and tools to achieve these objectives. The first 
module focuses on essential skills for 'Diagnosing and Improving Business 
Competitiveness', the second is about 'Managing Complexity in the Global 
DaimlerChrysler World'. 


Driving Success 

in a Global 

Company 

1 week 

DCU / IMD, 

Lausanne, 

Switzerland 


Executives Level 3/Band 94- 
95 (with potential) and above 
with an interest in cross- 
cultural leadership and its 
alignment with strategy and 
organisational dynamics 


The program focuses on cross-cultural leadership and general 
management issues facing DaimlerChrysler senior managers today and, to 
an even greater degree, in the future: strategic and entrepreneurial 
thinking and acting, leading cross-cultural, cross-functional teams in an 
increasingly uncertain and competitive global business environment, in a 
rather process- than structure-based organisation, with more "horizontal" 
interaction than hierarchical orders, more alacrity and rapid change. 

The program develops leadership and general management skills along 
DaimlerChrysler's leadership requirements interlinked with the strategic 
direction of the company. The program uses a wide range of tools, from 
case studies to role-play. Participants will work together during the week to 
capture the learning of the course and to develop an action plan for their 
own leadership role. 


Leadership in a 

Cross-cultural 

Environment 

1 week 

DCU/ IMD, 

Lausanne, 

Switzerland 


Executives level 3/Band 95 
(with potential) and above 
with more technical or 
functional tasks who are 
interested in a broader 
perspective of General 
Management issues 


Based on DaimlerChrysler corporate goals and future challenges, the 
program focuses on General Management issues. Certain best practice 
and case examples will be identified and the participants will draw their 
own conclusions of the lessons to be learned. Key issues of the course are 
understanding a global business environment as well as understanding 
what a global company is all about; designing processes for profitable 
growth throughout the value chain; impact of new technology; leadership 
requirements that create an entrepreneurial drive for innovation. 

The program develops general management skills necessary for driving 
success in a global company. It covers a wide range of themes and 
subjects tailored to DaimlerChrysler's needs, ranging from global strategy 
to external growth through mergers and acquisitions. 



Source: DCU 
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Non- 

Customised 

Programmes 


Target Group 


Brief Programme Description 


Driving Corporate 
Performance 
1 week 

Harvard Business 
School, Boston, 
MA, USA 


The course is suited for. 

• Chief financial officers; 

• Vice presidents of 
finance; and 

• Controllers. 

To facilitate strategy 
implementation throughout 
the organisation, COOs and 
general managers are also 
encouraged to attend 


Defining the Program's Objectives: 

The course examines innovative financial approaches that strengthen 
measurement, communication, and implementation of strategic plans. 
Participants develop a deeper understanding of a company's economics — 
from markets and competition to financial performance and strategy. 


Creating 
Corporate 
Advantage 
3 days 

Harvard Business 
School, Boston, 
MA, USA 


The course is designed for 
executives who are 
responsible for, or who are 
about to take over, the 
formulation and 
implementation of strategies 
that span multiple business 
units: 

• CEOs, CFOs, and 
COOs 

• Group or sector vice 
presidents 

• Directors of strategic 
planning 


Defining the Program's Objectives: 

CEO's and business unit managers who formulate and implement 
corporate strategy examine how success depends on combining a firm's 
assets, skills, and capabilities with its ability to leverage resources across 
traditional business boundaries. 


Advance 
Management 
Program 
9 weeks 

Harvard Business 
School, Boston, 
MA, USA 


Intended participants include: 

• Top corporate 
executives; 

• Managing directors of 
major business units; 
and 

• Senior executives 
poised to take on 
significant corporate 
leadership positions. 


Defining the Program's Objectives: 

AMP addresses the new scope of global opportunity by uniting some of the 
most respected minds in international business. The curriculum is 
extremely intensive and focuses on four key areas: 

• Developing effective corporate strategies; 

• Expanding management focus to a global perspective; 

• Exploring how the Internet and E-Commerce are changing the 
fundamentals of business; and 

• Understanding the unique responsibilities of the chief executive 
officer. 


The General 
Management 
Program 
7 weeks 

Harvard Business 
School, Boston, 
MA, USA 


The course is intended for a 
qualified audience of general 
managers from diverse 
business cultures around the 
world. General managers with 
demonstrated capabilities and 
proven leadership potential — 
from fast-track individuals in 
their first general 
management positions to 
seasoned professionals 
assuming major general 
management assignments— 
benefit equally from this 
highly interactive learning 
experience. 


Defining the Program's Objectives: 

Throughout the entire program, general managers develop a sound 

understanding of the crucial role they play in the success of their 

organisations. They return to their organisations armed with the knowledge 

and skills to competitively position their businesses well into the future. 

Specifically, participants: 

• Gain insight into critical business processes... the ways in which they 
evolve and interact to achieve strategic objectives and facilitate 
growth and long-term profitability. 

• Formulate an integrated business perspective 

• Strengthen leadership practices. 

• Sharpen strategic decision-making skills.. .based on a thorough 
understanding of how organisations operate, compete, and succeed. 

• Enhance teamwork and collaborative skills. Build and change 
organisational culture. 

• Formulate business strategies. ..that draw on existing capabilities and 
capitalise on new market opportunities. 

• Create results-oriented action plans... to ensure individual success 
and organisational competitiveness in today's global marketplace. 


Program Global 
Leadership 
6 weeks 

Harvard Business 
School, Boston, 
MA, USA 


The course is specifically 
intended for a select group of 
line executives, functional 
managers, and experienced 
professional specialists 
currently in or moving toward 
general management: 

• Vice president; 

• Subsidiary general 
manager; 

• Director; and 

• Unit manager. 


Defining the Program's Objectives: 

The Program for Global Leadership is organised around four core 
integrative, cross-functional courses. Led by senior faculty members, these 
courses provide a platform for building strong general management 
knowledge and skills — as well as a forum for engaging Harvard Business 
School's extensive pool of internal experts and external contacts to 
supplement the core course agenda. 

• Financial Strategy and Management Control 

• Strategic Management and Organisational Capabilities 

• Managing the Value Chain: Linking Technologies to Markets 

• Business, Government, and the International Economy 
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Strategic Human 
Resource 
Management 
1 week 

Harvard Business 
School, Boston, 
MA, USA 


The course is designed for 
senior line and HR executives 
who are in a position to 
directly influence 
organisational effectiveness 
and human resource policy 
issues critical to developing 
sustainable competitive 
advantage: 

• Chief executive officers; 

• Senior human resource 
executives; and 

• Other key line 
executives 


Defining the Program's Objectives: 

CEOs, human resource managers, and line managers examine how 
structures, systems, culture, and leadership impact organisational 
effectiveness - and explore the strategic role of the HR function 


Managerial Skills 

for International 

Business 

1 week 

INSEAD, 

Fontainebleau, 

France 


The seminar will most benefit 
senior managers currently or 
about to be posted abroad as 
well as those executives 
based at headquarters who 
co-ordinate or manage 
international activities. Typical 
participants 

• Division Manager 

• Marketing Director 

• Vice-President Human 
Resources 

• Regional Manager 

• international Project 
Leader 


Defining the Program's Objectives: 

To respond to these needs, INSEAD has adapted Managerial Skills for 
International Business to provide a unique multi-cultural learning 
experience directed at: 

• enhancing cross-cultural management skills 

• understanding international team-working 

• tackling global organisation management. 

Building upon INSEAD's distinctive competence in international teaching 
and research, the programme is an opportunity for managers grappling 
with management, leadership and organisation development in the multi- 
cultural environment of international business. 


Advanced 

Industrial 

Marketing Strategy 
2 weeks 
INSEAD, 
Fontainebleau, 
France 


Participants are from 
industrial corporations selling 
their products and services to 
other companies or 
institutions (business-to- 
business). 

The programme is also 
designed for senior general 
managers whose strategic 
role requires that they 
understand strategic 
industrial marketing and its 
underlying concepts in order 
to work effectively with their 
own marketing specialists. 


Defining the Program’s Objectives: 

Advanced Industrial Marketing Strategy (AIMS) is a unique programme 
which allows participants to experience first hand, in a complex marketing 
environment, the application and value of the concepts which underpin 
strategic market orientation. Through this experience, participants learn 
the strengths and weaknesses of the relevant tools. Another emphasis is 
learning about the dynamics of relationships between suppliers and their 
customers; how marketing should interface with research and 
development; how to motivate and manage field personnel and how to 
structure successful alliances between competitors - all key skills in 
industrial marketing today. 

Advanced Industrial Marketing Strategy is a programme designed for 
experienced marketing managers and other senior managers in strategic 
positions. As an advanced programme on industrial marketing for 
international managers it is unrivalled and therefore one of INSEAD's 
leadinq programmes. 


Advanced 

Management 

Programme 

4 weeks 

INSEAD, 

Fontainebleau, 

France 


Programme participants are 
all senior executives, head of 
a product division, a 
geographical region, a 
corporate function or a major 
joint venture who typically 
have at least 15 years’ 
management experience. 
Their budget responsibility 
typically ranges from US$50 
million to several billions and 
they are destined for the very 
top. They work in all sectors 
of the public and private 
economy, from large oil 
producers to e-commerce 
start-ups. 


Defining the Program's Objectives: 

The first week of the Advanced Management Programme (AMP)offers the 
opportunity to reflect on the trends and pressures shaping corporate life 
today, be it macro-changes at work in the world, and their impact on daily 
management tasks, or be it the pressures from organisations and their 
interpersonal aspects. 

Faced with such challenges, how is the modern corporation expected to 
respond? The central theme of the second week is about Value Creation: 
value for the customers, value for the shareholders, value for the various 
stakeholders. 

Given the increasing international competition and the need for modern 
corporations to create value, how does top management articulate a 
compelling vision, mobilise commitment, develop and grow the necessary 
resources and make change happen? The third week of the AMP is 
focused on Innovation and Change. 

By week 4, it is time to move from reflection to action. Based on what they 
learned and discussed during the first three weeks, participants are 
encouraged to propose concrete steps, both for their corporation ’s role in 
society, and for themselves. 


International 

Executive 

Programme 

6 weeks 

INSEAD, 

Fontainebleau, 

France 


Typical participants 

• Senior Functional 
Manager 

• Regional Manager 

• Business Unit Manager 

• Division Manager 

• Managing Director 


Defining the Program’s Objectives: 

The International Executive Programme is a complete management 
development experience, providing knowledge and skills on which to build 
in future years. Starting with a review of the fundamental disciplines, it 
advances to matters of policy and problem analysis, focusing on business 
strategy and financial analysis and control. The 'context' is provided by 
considering the relationships between business and the socio-political and 
economic environments 
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• Senior Engineer 


Some topics covered 

• Organisational behaviour 

• Human resource management 

• Financial management 

• Economic and political analysis 

• Marketing management 

• Accounting and control 

• Quality and operations management 

• Information technology 
Strategic management. 


International 

Manufacturing 

Programme 

2 weeks 

INSEAD, 

Fontainebleau, 

France 


Typical Participants 

• Director of 
Manufacturing 

• Technical Director 

• Production Manager 

• Plant Manager 

• Quality Manager 

• Vice President 
Operations 

• Logistics Manager 


Defining the Program's Objectives: 

Much of the accumulated wisdom in production management is now being 
critically re-examined. In addition, international competition for 
manufactured goods is intensifying and world-wide trade agreements 
demand constant review of a company’s global factory network. This 
programme analyses the changing rules and their implications for 
managers. The objective is to show that, correctly managed, 
manufacturing can create a formidable competitive weapon. 

• Some topics covered 

• Global trends in manufacturing 

• Managing global factory networks 

• Managing global supply chains 

• Managing quality 

• Managing www products and process technologies 

• Performance measures and activity-based costing 

• Work design and international industrial relations 

• Asia and the 'Pacific Century' 

• Green' manufacturing 

• Developing core manufacturing competence 


Strategy in 
Eastern Europe 
1 week 
INSEAD, 
Fontainebleau, 
France 


(Not in the current 
programme portfolio any 
more) 


(Not in the current programme portfolio any more) 


Strategic R&D 

Management 

1 week 

INSEAD, 

Fontainebleau, 

France 


Participants of the SRDM 

Programme usually come 

from the following categories: 

• corporate technical staff 

• laboratory directors 

• managers responsible 
for the product or project 
engineering 
organisations 

• heads of major research 
programmes 

• managers with a heavy 
stake in the successful 
running of the 
company's R&D effort 


Defining the Program's Objectives: 

INSEAD's Strategic R&D Management programme starts from the premise 
that the R&D function is wholly integrated into the rest of the firm’s 
activities. As such, we aim to improve the management of R&D as related 
to other functions, as well as within the function itself. Participants do not 
therefore review the basics of R&D management, but develop a business 
perspective. They tackle issues such as how to develop procedures to 
make R&D an efficient 'routine' and how to harness innovation to create a 
'learning organisation'. 


Hewlett Packard 

Management 

Academy 

2 weeks 

INSEAD, 

Fontainebleau, 

France 


Open to senior executives 
from Hewlett-Packard and 
their key channel partners 


Defining the Program's Objectives: 

These are exciting times for the computer channel. The relentless pace of 
technological change, increased competition, and evolving consumer 
requirements means that the firms which will survive and prosper must 
respond rapidly to changing opportunities and threats, focus their 
resources and be run efficiently. The INSEAD Hewlett-Packard 
Management Academy aims to provide firms with the skills necessary for 
managing in this complex environment. 

The programme topics include: 

• Corporate Strategy 

The strategy module focuses on three main issues. First, an understanding 
of the rules of the competitive game as they derive from the industry 
structure. Second, an emphasis on competitive advantage based on a 
detailed analysis of the value added chain in each segment. Third, the 
necessity to combine strategic focus with opportunistic flexibility as it is 
difficult to predict the future of the channel. 

• Marketing Strategy 

• Management of People 
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• Financial Management 

• Strategic Cost Management 

• Service Management 


Young Managers 

Programme 

3 weeks 

INSEAD, 

Fontainebleau, 

France 


Typical participants 

• Functional Manager 

• Product Manager 

• Country Manager 

• Engineer 

• Department Head 


The Young Managers Programme emphasises the interdisciplinary nature 
of business and exposes confirmed managers to problems outside their 
own specialised fields. It will equip participants with the knowledge and 
skills to exercise higher managerial responsibilities successfully. 

Some topics covered: 

• Finance 

• Organisational Behaviour 

• Marketing 

• Operations and Service Management 

• Management Accounting 

• Strategy 

• International Management and Entrepreneurship 


International 
Program for Board 
Members 
3 days 

IMD, Lausanne, 
Switzerland 


Level: Members of the 
Board, experienced and 
newly appointed. 

Function: Represent a broad 
cross-section of 
manufacturing and service 
companies 


Program Scope and Objectives 

The International Program for Board Members focuses on the processes 
by which corporations are directed, controlled, and accounted for at the 
highest levels, and on the management challenges which the board must 
address. 

Program Overview 

The International Program for Board Members addresses board member 
responsibilities under three broad headings: first, how can the board 
properly assess the management challenges affecting the company; 
second, how should the board relate to top management and the CEO, 
given the company's particular circumstances; and third, how are high- 
performing boards structured and how do they function to meet these 
challenges? 


Program for 
Executive 
Development 
1 0 weeks 
IMD, Lausanne, 
Switzerland 


Level: High potential mid- 
level managers 
Function: General 
Management, Finance, 
Marketing, IT, Manufacturing, 
Strategic Planning 


Defining the Program's Objectives: 

The Program for Executive Development brings together high potential, 
upper and mid-level managers from all over the world and provides them 
with a learning environment for the exploration of contemporary 
management issues. 

The program is about awareness and discovery. It is a time for challenge 
and a time for reflection. It is a new beginning. 

The focus of module one is on leading the business. The first five weeks of 
PED address the challenges of building the high-performance business 
team, meeting the simultaneous demands for quality, low cost, rapid 
response, stimulating growth, accelerating organisational learning, and 
integrating business processes for optimal value. 

Module two is designed to help you take charge of a business or 
corporation competing globally. To help better balance the increasing 
demands of senior leadership, module two focuses on you as a strategist, 
an entrepreneur, a person who knows how to get things done, a manager 
of interfaces, a citizen and a family member. 


Orchestrating 
Winning 
Performance 
1 week 

IMD, Lausanne, 
Switzerland 


Level: Leading managers of 
various levels 
Function: General 
management, Marketing, 
Technology, Operations, 
Finance, Human Resources 


Defining the Program's Objectives: 

OWP is an intensive exposure to state-of-the-art thinking on the latest 
management issues. What really sets OWP apart is that it breaks with 
most executive education traditions by allowing you to 'design' your own 
program and tailor it to meet your organisation's specific needs. Another 
unique feature is that every IMD Faculty member takes part in the 
program, so you will have on-the-spot access to a concentration of some 
of the world's leading management expertise on the realities that are 
shaping today's international business environment. And all this by 
spending only four working days away from your office. OWP has become 
a not-to-be-missed event on the annual agenda of hundreds of fast-track 
executives world-wide. Why not join the winning team? 

You will discover that OWP is: 

• Focused on practical, up-to-the-minute issues that have real take- 
home value 

• Designed to stimulate creative 'out of the box thinking' 

• A unique networking opportunity with over 350 managers from world- 
class companies 

• Equally valuable for individuals as well as corporate teams 

At OWP, we won't tell you how to run your business - but we will give you 
the latest knowledge and tools to run it better. 


Mobilising People 
2 weeks 
IMD, Lausanne, 
Switzerland 


Level: Managers from 
different functional 
backgrounds, experiences 
and levels of responsibility. 
Function: Marketing/Sales, 


Defining the Program's Objectives: 

MP is a two-week program for managers who want to improve their ability 
to influence others, particularly their ability to mobilise human talent to 
meet corporate, divisional, team, or project objectives. 

The pathway I 
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General Management, 
Finance, Technology, 
Operations, Strategic 
Planning, HR 


• Discover your true talents, capabilities, interests and motivations 

• Identify the obstacles - often self-imposed - in the way of your best 
self-expression 

• Open yourself to many and diverse learning experiences and inputs 

• Study relevant role models 

• Be prepared to challenge all you have previously believed about your, 
and others', strengths and limitations 

• Let your desired future cause your present behaviour - challenge the 
influence of the past 


Managing for 
Marketing 
Success 
1 x 2 weeks 
IMD, Lausanne, 
Switzerland 


Level: Managers involved 
with marketing, sales, or e- 
commerce within consumer 
goods, industrial and sen/ice 
industries. 


Defining the Program's Objectives: 

MNE [the successor of MMs] examines how to make an organisation more 
market-driven and increase the efficiency of its marketing processes. 
Explores the changing role of the marketing manager's functions; the 
growing strength of major retailers; the enhancement of product offerings 
with service and the challenges of internet and global marketing; and the 
need to transform marketing for effectiveness in the increasingly 
networked environment. 


HR Leadership in 
Global 

Organisations 
1 week 

IMD, Lausanne, 
Switzerland 


(Not in the current 
! programme portfolio any 
more) 


(Not in the current programme portfolio any more) 


Managing 

Corporate 

Resources 

3 weeks 

IMD, Lausanne, 

Switzerland 


Level: General management 
responsibility for a business 
unit, division, region or 
country 

Function: General 
Management, Marketing, 
Finance, Operations, R&D, 
Human Resources 


Defining the Program's Objectives: 

MCR is a three-week intensive, project-based program for managers 
involved in the co-ordination and allocation of corporate resources inside 
and across business units or divisions. The program combines personal 
skill building with an opportunity to reflect on and discuss broader topical 
business issues with peers from around the world. It will help you to better 
understand and integrate the various functions of your business unit and 
manage change in the wider context of an overall corporate strategy. 



Source: 



“http://www.exed.hbs.edu/programs"; “http://www.insead.fr/executives”; “http://www.imd. ch r 
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